2024-09-13T07:00:59.150 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T07:00:59.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T07:00:59.257 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900909' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/EVN3lNe4Dh4k/3ZuykzC+Pnw87DwnhID5eEtcodpIhz5vtk789fTzLMtByYtGd+V7kkP2FqRyY2QJ5pIt/iU= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPQOOEitnzJ/qSjJcwV/cK+1PcUH6n8Humuo1Spa2wxi3YcXOrGsv77v244sD0K965lw7waQj4ULutAjKSvWMVc= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T07:00:59.257 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T07:00:59.258 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T07:00:59.258 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T07:00:59.260 INFO:teuthology.task.internal:Checking packages... 2024-09-13T07:00:59.283 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T07:00:59.283 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T07:00:59.283 INFO:teuthology.packaging:ref: None 2024-09-13T07:00:59.283 INFO:teuthology.packaging:tag: None 2024-09-13T07:00:59.283 INFO:teuthology.packaging:branch: reef 2024-09-13T07:00:59.284 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T07:00:59.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T07:00:59.586 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843.g2c01458a 2024-09-13T07:00:59.588 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T07:00:59.594 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T07:00:59.594 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T07:00:59.601 INFO:teuthology.task.internal:Saving configuration 2024-09-13T07:00:59.614 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T07:00:59.623 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T07:00:59.657 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 06:56:07.493259', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/EVN3lNe4Dh4k/3ZuykzC+Pnw87DwnhID5eEtcodpIhz5vtk789fTzLMtByYtGd+V7kkP2FqRyY2QJ5pIt/iU='} 2024-09-13T07:00:59.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 06:56:07.491371', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPQOOEitnzJ/qSjJcwV/cK+1PcUH6n8Humuo1Spa2wxi3YcXOrGsv77v244sD0K965lw7waQj4ULutAjKSvWMVc='} 2024-09-13T07:00:59.687 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T07:00:59.696 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-13T07:00:59.696 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-13T07:00:59.696 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T07:00:59.777 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc0821415a0>, signals=[15]) 2024-09-13T07:00:59.778 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T07:00:59.788 INFO:teuthology.task.internal:Opening connections... 2024-09-13T07:00:59.788 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:00:59.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:00:59.891 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:00:59.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:00:59.992 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T07:01:00.000 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-13T07:01:00.018 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-13T07:01:00.019 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-13T07:01:00.077 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-13T07:01:00.078 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T07:01:00.078 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-13T07:01:00.078 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T07:01:00.078 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T07:01:00.078 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T07:01:00.078 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-13T07:01:00.106 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-09-13T07:01:00.125 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-09-13T07:01:00.125 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-09-13T07:01:00.184 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T07:01:00.185 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-09-13T07:01:00.185 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T07:01:00.185 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T07:01:00.185 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T07:01:00.185 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-09-13T07:01:00.215 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T07:01:00.228 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T07:01:00.241 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T07:01:00.241 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-13T07:01:00.244 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-09-13T07:01:00.260 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T07:01:00.271 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T07:01:00.271 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-13T07:01:00.301 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-09-13T07:01:00.327 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T07:01:00.395 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T07:01:00.413 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T07:01:00.413 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T07:01:00.414 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T07:01:00.414 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T07:01:00.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:01:00.431 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-13T07:01:00.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:01:00.432 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-13T07:01:00.488 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-13T07:01:00.488 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-13T07:01:00.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-13T07:01:00.490 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-503.el9.x86_64 2024-09-13T07:01:00.490 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-503.el9.x86_64 2024-09-13T07:01:00.490 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-09-13T07:01:03.335 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.2 MB 00:01 2024-09-13T07:01:03.384 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.2 MB 00:01 2024-09-13T07:01:05.511 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-13T07:01:09.995 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-13T07:01:11.671 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-13T07:01:12.408 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-13T07:01:18.080 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-13T07:01:20.187 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T07:01:20.187 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:01:20.531 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 174 kB/s | 23 kB 00:00 2024-09-13T07:01:21.001 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-13T07:01:21.468 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 32 MB/s | 35 MB 00:01 2024-09-13T07:01:21.735 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-13T07:01:21.735 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:01:21.735 INFO:teuthology.orchestra.run.smithi031.stdout:Total 55 MB/s | 79 MB 00:01 2024-09-13T07:01:22.448 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:01:22.839 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:01:22.839 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:01:23.782 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:19 2024-09-13T07:01:23.858 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:01:23.859 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:01:27.228 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T07:01:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T07:01:29.424 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-13T07:01:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T07:01:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T07:01:31.141 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-13T07:01:31.971 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-13T07:01:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T07:01:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:01:37.435 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T07:01:39.458 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T07:01:39.458 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 79 M 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 126 M 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:01:39.903 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 76 kB/s | 23 kB 00:00 2024-09-13T07:01:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.8 MB/s | 16 MB 00:05 2024-09-13T07:01:49.227 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.1 MB/s | 29 MB 00:09 2024-09-13T07:02:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:02:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:02:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:02:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T07:02:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T07:02:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:02:20.681 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:02:21.031 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T07:02:21.121 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T07:02:21.121 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T07:02:21.121 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T07:02:21.122 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T07:02:21.122 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T07:02:21.122 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T07:02:21.122 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T07:02:21.122 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T07:02:21.180 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T07:02:21.180 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T07:02:21.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-13T07:02:21.211 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T07:02:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:02:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T07:02:21.214 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T07:02:21.214 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-505.el9.x86_64 2024-09-13T07:02:21.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T07:02:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 814 kB/s | 35 MB 00:43 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.8 MB/s | 79 MB 00:43 2024-09-13T07:02:23.231 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T07:02:23.253 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-13T07:02:23.254 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-13T07:02:23.269 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.LTYnc4pgYm 2024-09-13T07:02:23.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LTYnc4pgYm 2024-09-13T07:02:23.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.LTYnc4pgYm 2024-09-13T07:02:23.478 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.LTYnc4pgYm is 6KB 2024-09-13T07:02:23.527 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.LTYnc4pgYm 2024-09-13T07:02:23.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T07:02:23.604 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T07:02:23.605 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T07:02:23.605 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T07:02:23.605 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T07:02:23.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T07:02:23.725 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:02:23.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-13T07:02:24.071 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:02:24.071 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:02:25.028 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:02:25.029 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:02:27.468 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:02:28.217 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T07:02:28.317 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T07:02:31.119 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T07:02:31.253 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T07:02:36.069 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T07:02:36.186 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:02:53.760 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T07:02:53.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T07:02:53.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:03:12.189 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-13T07:03:19.233 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:03:19.272 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T07:03:20.543 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:03:20.872 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T07:03:20.997 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T07:03:20.998 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T07:03:20.998 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T07:03:20.998 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T07:03:20.998 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T07:03:20.998 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T07:03:20.998 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T07:03:20.998 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T07:03:21.057 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T07:03:21.057 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T07:03:21.057 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 82.el9 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8274670 2024-09-13T07:03:21.099 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T07:03:21.100 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T07:03:21.101 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2024-09-13T07:03:21.101 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:03:21.101 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T07:03:21.101 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T07:03:21.102 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 5.14.0-505.el9.x86_64 2024-09-13T07:03:21.102 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T07:03:21.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T07:03:21.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:03:21.768 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-09-13T07:03:23.087 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T07:03:23.117 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-09-13T07:03:23.118 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-13T07:03:23.134 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.sl9YIBkZFR 2024-09-13T07:03:23.134 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.sl9YIBkZFR 2024-09-13T07:03:23.201 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.sl9YIBkZFR 2024-09-13T07:03:23.330 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.sl9YIBkZFR is 6KB 2024-09-13T07:03:23.378 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.sl9YIBkZFR 2024-09-13T07:03:23.393 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T07:03:23.455 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T07:03:23.456 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T07:03:23.456 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T07:03:23.456 INFO:teuthology.orchestra.run.smithi184.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T07:03:23.457 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T07:03:23.613 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-09-13T07:03:39.581 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T07:03:51.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:03:52.275 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-13T07:03:52.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T07:03:52.728 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T07:03:52.729 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-13T07:03:52.744 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-505.el9.x86_64 2024-09-13T07:03:52.744 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T07:03:52.744 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T07:03:52.744 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T07:03:53.616 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T07:03:53.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-13T07:03:53.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:03:53.745 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T07:03:53.745 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T07:03:53.807 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-13T07:03:53.838 DEBUG:teuthology.parallel:result is None 2024-09-13T07:04:12.093 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-09-13T07:04:21.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-13T07:04:21.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:05:21.106 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T07:05:33.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-13T07:05:33.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:05:33.563 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-13T07:05:33.932 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-13T07:05:33.933 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T07:05:33.933 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-09-13T07:05:33.951 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-505.el9.x86_64 2024-09-13T07:05:33.951 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T07:05:33.951 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T07:05:33.951 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T07:05:34.953 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T07:05:34.953 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T07:05:35.316 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2024-09-13T07:05:35.336 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:35.336 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T07:05:35.343 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T07:05:35.343 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T07:05:35.347 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T07:05:35.365 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T07:05:35.372 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T07:05:35.380 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T07:05:35.380 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T07:05:35.408 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T07:05:35.439 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T07:05:35.446 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T07:05:35.446 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T07:05:35.481 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T07:05:35.518 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T07:05:35.520 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T07:05:35.529 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T07:05:35.532 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T07:05:35.534 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T07:05:35.541 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T07:05:35.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T07:05:35.574 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T07:05:35.605 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T07:05:35.614 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T07:05:35.614 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T07:05:35.644 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T07:05:35.663 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T07:05:35.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T07:05:35.792 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T07:05:35.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T07:05:35.917 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:05:35.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T07:05:35.985 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T07:05:36.038 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T07:05:36.082 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T07:05:36.176 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T07:05:36.212 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:05:36.212 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T07:05:36.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-13T07:05:36.283 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-09-13T07:05:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T07:05:36.366 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T07:05:36.711 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T07:05:36.719 INFO:teuthology.task.internal:Starting timer... 2024-09-13T07:05:36.719 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T07:05:36.861 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T07:05:36.870 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-13T07:05:36.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-13T07:05:36.925 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-13T07:05:36.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-09-13T07:05:36.965 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-09-13T07:05:36.967 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T07:05:36.967 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-13T07:05:37.011 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-13T07:05:37.011 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-09-13T07:05:37.031 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-09-13T07:05:37.031 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-09-13T07:05:37.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T07:05:37.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:05:37.081 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T07:05:37.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:05:37.108 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T07:05:37.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-13T07:05:37.158 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-09-13T07:05:37.193 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T07:05:37.203 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T07:05:37.214 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T07:05:37.215 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T07:09:22.058 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-09-13T07:09:22.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T07:09:22.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:09:22.147 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-13T07:09:22.233 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T07:09:22.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-13T07:09:22.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T07:09:22.320 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-09-13T07:09:22.397 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-09-13T07:09:22.397 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T07:09:22.407 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T07:09:22.407 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T07:09:22.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T07:09:22.410 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T07:09:22.410 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T07:09:22.440 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T07:09:22.454 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T07:09:22.466 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T07:09:22.479 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-13T07:09:22.480 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T07:09:22.490 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-13T07:09:22.504 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-09-13T07:09:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T07:09:22.515 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-09-13T07:09:22.521 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T07:09:22.529 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T07:09:22.543 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T07:09:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-13T07:09:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T07:09:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-13T07:09:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.604 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-13T07:09:22.608 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T07:09:22.608 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-13T07:09:22.608 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.608 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.608 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.608 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:09:22.609 INFO:teuthology.run_tasks:Running task install... 2024-09-13T07:09:22.617 DEBUG:teuthology.task.install:project ceph 2024-09-13T07:09:22.617 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T07:09:22.617 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T07:09:22.617 INFO:teuthology.task.install:Using flavor: default 2024-09-13T07:09:22.622 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T07:09:22.622 INFO:teuthology.task.install:extra packages: [] 2024-09-13T07:09:22.623 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'tag': None, 'wait_for_package': False} 2024-09-13T07:09:22.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T07:09:22.626 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'tag': None, 'wait_for_package': False} 2024-09-13T07:09:22.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T07:09:22.794 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/ 2024-09-13T07:09:22.794 INFO:teuthology.task.install.rpm:Package version is 18.2.4-820.g48a7de7e 2024-09-13T07:09:22.811 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/ 2024-09-13T07:09:22.811 INFO:teuthology.task.install.rpm:Package version is 18.2.4-820.g48a7de7e 2024-09-13T07:09:22.921 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T07:09:22.922 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:09:22.922 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T07:09:22.935 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T07:09:22.935 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:09:22.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T07:09:22.948 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T07:09:22.949 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T07:09:22.962 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T07:09:22.962 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T07:09:23.015 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T07:09:23.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T07:09:23.092 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T07:09:23.105 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T07:09:23.156 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-09-13T07:09:23.158 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-09-13T07:09:23.171 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-13T07:09:23.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-13T07:09:23.390 INFO:teuthology.orchestra.run.smithi184.stdout:45 files removed 2024-09-13T07:09:23.413 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-13T07:09:23.416 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw 2024-09-13T07:09:23.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-09-13T07:09:23.967 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 290 kB/s | 77 kB 00:00 2024-09-13T07:09:24.012 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 286 kB/s | 77 kB 00:00 2024-09-13T07:09:24.259 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T07:09:24.296 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T07:09:24.534 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T07:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T07:09:25.234 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-13T07:09:26.488 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-13T07:09:28.869 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 20 MB 00:02 2024-09-13T07:09:28.922 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-13T07:09:33.253 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-13T07:09:34.498 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-13T07:09:34.820 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 19 kB 00:00 2024-09-13T07:09:35.766 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-13T07:09:35.957 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 19 kB 00:00 2024-09-13T07:09:36.761 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-13T07:09:41.122 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T07:09:42.567 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 9.6 kB/s | 1.7 kB 00:00 2024-09-13T07:09:43.489 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 7.7 M 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.3 M 2024-09-13T07:09:43.497 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 5.1 M 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 18 M 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 25 k 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 711 k 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T07:09:43.498 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 478 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.5 M 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 45 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 129 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 162 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 321 k 2024-09-13T07:09:43.499 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 299 k 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 99 k 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout:Install 28 Packages 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2024-09-13T07:09:43.500 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:09:43.501 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 53 M 2024-09-13T07:09:43.501 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:09:44.431 INFO:teuthology.orchestra.run.smithi184.stdout:(1/30): ceph-radosgw-18.2.4-820.g48a7de7e.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-13T07:09:44.498 INFO:teuthology.orchestra.run.smithi184.stdout:(2/30): ceph-base-18.2.4-820.g48a7de7e.el9.x86_ 6.7 MB/s | 5.1 MB 00:00 2024-09-13T07:09:44.523 INFO:teuthology.orchestra.run.smithi184.stdout:(3/30): ceph-selinux-18.2.4-820.g48a7de7e.el9.x 274 kB/s | 25 kB 00:00 2024-09-13T07:09:44.574 INFO:teuthology.orchestra.run.smithi184.stdout:(4/30): libradosstriper1-18.2.4-820.g48a7de7e.e 9.3 MB/s | 478 kB 00:00 2024-09-13T07:09:44.607 INFO:teuthology.orchestra.run.smithi184.stdout:(5/30): libcephfs2-18.2.4-820.g48a7de7e.el9.x86 6.4 MB/s | 711 kB 00:00 2024-09-13T07:09:44.799 INFO:teuthology.orchestra.run.smithi184.stdout:(6/30): ceph-common-18.2.4-820.g48a7de7e.el9.x8 17 MB/s | 18 MB 00:01 2024-09-13T07:09:44.825 INFO:teuthology.orchestra.run.smithi184.stdout:(7/30): python3-ceph-argparse-18.2.4-820.g48a7d 207 kB/s | 45 kB 00:00 2024-09-13T07:09:44.850 INFO:teuthology.orchestra.run.smithi184.stdout:(8/30): python3-ceph-common-18.2.4-820.g48a7de7 2.5 MB/s | 129 kB 00:00 2024-09-13T07:09:44.876 INFO:teuthology.orchestra.run.smithi184.stdout:(9/30): python3-cephfs-18.2.4-820.g48a7de7e.el9 3.2 MB/s | 162 kB 00:00 2024-09-13T07:09:44.901 INFO:teuthology.orchestra.run.smithi184.stdout:(10/30): python3-rados-18.2.4-820.g48a7de7e.el9 6.2 MB/s | 321 kB 00:00 2024-09-13T07:09:44.934 INFO:teuthology.orchestra.run.smithi184.stdout:(11/30): python3-rbd-18.2.4-820.g48a7de7e.el9.x 5.0 MB/s | 299 kB 00:00 2024-09-13T07:09:44.959 INFO:teuthology.orchestra.run.smithi184.stdout:(12/30): python3-rgw-18.2.4-820.g48a7de7e.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-13T07:09:45.022 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:09:45.025 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:09:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:09:45.025 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:09:45.025 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 7.7 M 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.3 M 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 5.1 M 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 18 M 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 25 k 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T07:09:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 711 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 478 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.5 M 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 45 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 129 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 162 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T07:09:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 321 k 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 299 k 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 99 k 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout:Install 28 Packages 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-09-13T07:09:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:09:45.029 INFO:teuthology.orchestra.run.smithi184.stdout:(13/30): librgw2-18.2.4-820.g48a7de7e.el9.x86_6 9.9 MB/s | 4.5 MB 00:00 2024-09-13T07:09:45.029 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 53 M 2024-09-13T07:09:45.029 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:09:45.102 INFO:teuthology.orchestra.run.smithi184.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 639 kB/s | 107 kB 00:00 2024-09-13T07:09:45.135 INFO:teuthology.orchestra.run.smithi184.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-13T07:09:45.160 INFO:teuthology.orchestra.run.smithi184.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 798 kB/s | 160 kB 00:00 2024-09-13T07:09:45.211 INFO:teuthology.orchestra.run.smithi184.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 848 kB/s | 42 kB 00:00 2024-09-13T07:09:45.253 INFO:teuthology.orchestra.run.smithi184.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.6 MB/s | 111 kB 00:00 2024-09-13T07:09:45.286 INFO:teuthology.orchestra.run.smithi184.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-13T07:09:45.320 INFO:teuthology.orchestra.run.smithi184.stdout:(20/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.5 MB/s | 308 kB 00:00 2024-09-13T07:09:45.345 INFO:teuthology.orchestra.run.smithi184.stdout:(21/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2024-09-13T07:09:45.370 INFO:teuthology.orchestra.run.smithi184.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 532 kB/s | 26 kB 00:00 2024-09-13T07:09:45.404 INFO:teuthology.orchestra.run.smithi184.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-13T07:09:45.438 INFO:teuthology.orchestra.run.smithi184.stdout:(24/30): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-13T07:09:45.480 INFO:teuthology.orchestra.run.smithi184.stdout:(25/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-13T07:09:45.505 INFO:teuthology.orchestra.run.smithi184.stdout:(26/30): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-13T07:09:45.564 INFO:teuthology.orchestra.run.smithi184.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-13T07:09:45.631 INFO:teuthology.orchestra.run.smithi184.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-13T07:09:45.634 INFO:teuthology.orchestra.run.smithi031.stdout:(1/30): ceph-base-18.2.4-820.g48a7de7e.el9.x86_ 12 MB/s | 5.1 MB 00:00 2024-09-13T07:09:45.710 INFO:teuthology.orchestra.run.smithi031.stdout:(2/30): ceph-selinux-18.2.4-820.g48a7de7e.el9.x 336 kB/s | 25 kB 00:00 2024-09-13T07:09:45.818 INFO:teuthology.orchestra.run.smithi031.stdout:(3/30): libcephfs2-18.2.4-820.g48a7de7e.el9.x86 6.4 MB/s | 711 kB 00:00 2024-09-13T07:09:45.889 INFO:teuthology.orchestra.run.smithi184.stdout:(29/30): librbd1-18.2.4-820.g48a7de7e.el9.x86_6 9.3 MB/s | 3.0 MB 00:00 2024-09-13T07:09:45.910 INFO:teuthology.orchestra.run.smithi031.stdout:(4/30): libradosstriper1-18.2.4-820.g48a7de7e.e 5.1 MB/s | 478 kB 00:00 2024-09-13T07:09:45.973 INFO:teuthology.orchestra.run.smithi184.stdout:(30/30): librados2-18.2.4-820.g48a7de7e.el9.x86 7.1 MB/s | 3.3 MB 00:00 2024-09-13T07:09:45.977 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:09:45.977 INFO:teuthology.orchestra.run.smithi184.stdout:Total 21 MB/s | 53 MB 00:02 2024-09-13T07:09:45.994 INFO:teuthology.orchestra.run.smithi031.stdout:(5/30): ceph-radosgw-18.2.4-820.g48a7de7e.el9.x 9.8 MB/s | 7.7 MB 00:00 2024-09-13T07:09:46.047 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:09:46.052 INFO:teuthology.orchestra.run.smithi031.stdout:(6/30): python3-ceph-argparse-18.2.4-820.g48a7d 775 kB/s | 45 kB 00:00 2024-09-13T07:09:46.081 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:09:46.082 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:09:46.128 INFO:teuthology.orchestra.run.smithi031.stdout:(7/30): librgw2-18.2.4-820.g48a7de7e.el9.x86_64 21 MB/s | 4.5 MB 00:00 2024-09-13T07:09:46.153 INFO:teuthology.orchestra.run.smithi031.stdout:(8/30): python3-ceph-common-18.2.4-820.g48a7de7 1.3 MB/s | 129 kB 00:00 2024-09-13T07:09:46.178 INFO:teuthology.orchestra.run.smithi031.stdout:(9/30): python3-cephfs-18.2.4-820.g48a7de7e.el9 3.1 MB/s | 162 kB 00:00 2024-09-13T07:09:46.204 INFO:teuthology.orchestra.run.smithi031.stdout:(10/30): python3-rados-18.2.4-820.g48a7de7e.el9 6.3 MB/s | 321 kB 00:00 2024-09-13T07:09:46.229 INFO:teuthology.orchestra.run.smithi031.stdout:(11/30): python3-rbd-18.2.4-820.g48a7de7e.el9.x 5.8 MB/s | 299 kB 00:00 2024-09-13T07:09:46.254 INFO:teuthology.orchestra.run.smithi031.stdout:(12/30): python3-rgw-18.2.4-820.g48a7de7e.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-13T07:09:46.438 INFO:teuthology.orchestra.run.smithi031.stdout:(13/30): ceph-common-18.2.4-820.g48a7de7e.el9.x 15 MB/s | 18 MB 00:01 2024-09-13T07:09:46.480 INFO:teuthology.orchestra.run.smithi031.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 427 kB/s | 107 kB 00:00 2024-09-13T07:09:46.530 INFO:teuthology.orchestra.run.smithi031.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 493 kB/s | 45 kB 00:00 2024-09-13T07:09:46.555 INFO:teuthology.orchestra.run.smithi031.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 533 kB/s | 160 kB 00:00 2024-09-13T07:09:46.581 INFO:teuthology.orchestra.run.smithi031.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-13T07:09:46.623 INFO:teuthology.orchestra.run.smithi031.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-13T07:09:46.648 INFO:teuthology.orchestra.run.smithi031.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 458 kB/s | 42 kB 00:00 2024-09-13T07:09:46.690 INFO:teuthology.orchestra.run.smithi031.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.0 MB/s | 111 kB 00:00 2024-09-13T07:09:46.708 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:09:46.709 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:09:46.715 INFO:teuthology.orchestra.run.smithi031.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-13T07:09:46.740 INFO:teuthology.orchestra.run.smithi031.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T07:09:46.766 INFO:teuthology.orchestra.run.smithi031.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-13T07:09:46.799 INFO:teuthology.orchestra.run.smithi031.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-13T07:09:46.833 INFO:teuthology.orchestra.run.smithi031.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-13T07:09:46.866 INFO:teuthology.orchestra.run.smithi031.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-13T07:09:46.925 INFO:teuthology.orchestra.run.smithi031.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-13T07:09:47.017 INFO:teuthology.orchestra.run.smithi031.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-13T07:09:47.075 INFO:teuthology.orchestra.run.smithi031.stdout:(29/30): librados2-18.2.4-820.g48a7de7e.el9.x86 16 MB/s | 3.3 MB 00:00 2024-09-13T07:09:47.159 INFO:teuthology.orchestra.run.smithi031.stdout:(30/30): librbd1-18.2.4-820.g48a7de7e.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-13T07:09:47.163 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:09:47.164 INFO:teuthology.orchestra.run.smithi031.stdout:Total 25 MB/s | 53 MB 00:02 2024-09-13T07:09:47.234 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:09:47.273 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:09:47.273 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:09:47.914 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:09:47.915 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:09:48.110 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:09:48.190 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T07:09:48.227 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T07:09:48.277 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T07:09:48.641 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T07:09:48.702 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T07:09:48.794 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T07:09:48.878 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T07:09:48.903 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T07:09:48.974 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T07:09:49.011 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T07:09:49.054 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 9/32 2024-09-13T07:09:49.120 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T07:09:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T07:09:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T07:09:49.335 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T07:09:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:09:49.371 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T07:09:49.421 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T07:09:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T07:09:49.452 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T07:09:49.479 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T07:09:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T07:09:49.515 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T07:09:49.548 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T07:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T07:09:49.846 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T07:09:49.894 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T07:09:49.940 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T07:09:49.967 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T07:09:49.996 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T07:09:49.999 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 21/32 2024-09-13T07:09:50.043 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T07:09:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T07:09:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T07:09:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T07:09:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T07:09:50.281 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T07:09:50.325 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 9/32 2024-09-13T07:09:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T07:09:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T07:09:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T07:09:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T07:09:50.491 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T07:09:50.534 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T07:09:50.603 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 25/32 2024-09-13T07:09:50.608 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T07:09:50.632 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 26/32 2024-09-13T07:09:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T07:09:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T07:09:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T07:09:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T07:09:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T07:09:50.827 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T07:09:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T07:09:51.189 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T07:09:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T07:09:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 21/32 2024-09-13T07:09:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T07:09:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T07:09:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T07:09:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T07:09:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 25/32 2024-09-13T07:09:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 26/32 2024-09-13T07:09:52.799 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T07:09:52.845 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T07:09:53.371 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T07:09:53.407 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T07:09:53.505 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T07:09:53.505 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T07:09:53.505 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T07:09:53.505 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:09:53.549 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T07:09:54.215 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T07:09:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T07:09:54.840 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T07:09:54.876 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T07:09:54.976 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T07:09:54.977 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T07:09:54.977 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T07:09:54.977 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:09:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T07:10:03.393 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T07:10:03.393 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-09-13T07:10:03.394 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-09-13T07:10:03.394 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-09-13T07:10:03.394 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-09-13T07:10:03.394 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-09-13T07:10:03.394 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-09-13T07:10:03.394 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-09-13T07:10:03.394 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:04.130 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T07:10:04.156 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T07:10:04.156 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:04.156 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T07:10:04.156 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T07:10:04.156 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T07:10:04.157 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:04.158 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T07:10:04.233 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T07:10:04.233 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 1/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 3/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 4/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T07:10:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 8/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 9/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 12/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 13/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T07:10:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T07:10:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 31/32 2024-09-13T07:10:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T07:10:05.451 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-13T07:10:05.451 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-13T07:10:05.451 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-13T07:10:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-13T07:10:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-13T07:10:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-13T07:10:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-13T07:10:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T07:10:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T07:10:06.268 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:06.268 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T07:10:06.269 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T07:10:06.269 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T07:10:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T07:10:06.358 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T07:10:06.358 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.427 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T07:10:06.428 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:06.429 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:06.764 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-test 2024-09-13T07:10:07.339 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:26 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:07.478 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:07.478 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:07.478 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:10:07.478 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:07.478 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 40 M 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 166 k 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-09-13T07:10:07.479 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:07.480 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 40 M 2024-09-13T07:10:07.480 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 164 M 2024-09-13T07:10:07.480 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 1/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 3/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 4/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 8/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 9/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T07:10:07.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 12/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 13/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T07:10:07.577 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T07:10:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T07:10:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T07:10:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T07:10:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T07:10:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T07:10:07.579 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T07:10:07.579 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T07:10:07.579 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 31/32 2024-09-13T07:10:07.948 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): libcephsqlite-18.2.4-820.g48a7de7e.el9.x 785 kB/s | 166 kB 00:00 2024-09-13T07:10:07.973 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-13T07:10:07.998 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T07:10:08.764 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:08.767 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:09.032 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): ceph-test-18.2.4-820.g48a7de7e.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-13T07:10:09.033 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:09.034 INFO:teuthology.orchestra.run.smithi184.stdout:Total 26 MB/s | 40 MB 00:01 2024-09-13T07:10:09.061 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:09.080 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:09.080 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:09.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-09-13T07:10:09.407 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:09.407 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:10:09.725 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:09.864 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:09.864 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 40 M 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 166 k 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-13T07:10:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:09.866 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 40 M 2024-09-13T07:10:09.866 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 164 M 2024-09-13T07:10:09.866 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:09.881 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:10:09.958 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T07:10:09.997 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T07:10:10.025 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T07:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-09-13T07:10:10.215 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-13T07:10:10.241 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): libcephsqlite-18.2.4-820.g48a7de7e.el9.x 690 kB/s | 166 kB 00:00 2024-09-13T07:10:11.282 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-18.2.4-820.g48a7de7e.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-13T07:10:11.284 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:11.284 INFO:teuthology.orchestra.run.smithi031.stdout:Total 28 MB/s | 40 MB 00:01 2024-09-13T07:10:11.313 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:10:11.335 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:10:11.335 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:10:11.684 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:10:11.684 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:10:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:10:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T07:10:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T07:10:12.379 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T07:10:13.733 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T07:10:13.806 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T07:10:15.871 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T07:10:15.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 1/4 2024-09-13T07:10:15.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2/4 2024-09-13T07:10:15.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T07:10:16.211 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T07:10:16.211 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:16.211 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:16.211 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:16.212 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:16.212 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T07:10:16.212 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T07:10:16.212 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:16.212 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T07:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T07:10:16.510 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph 2024-09-13T07:10:17.083 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:36 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:17.218 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 6.5 k 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 2.1 M 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 1.5 M 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 246 k 2024-09-13T07:10:17.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.7 M 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 17 M 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T07:10:17.221 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T07:10:17.222 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:17.223 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:17.223 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:17.223 INFO:teuthology.orchestra.run.smithi184.stdout:Install 33 Packages 2024-09-13T07:10:17.223 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:17.223 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 29 M 2024-09-13T07:10:17.223 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 104 M 2024-09-13T07:10:17.223 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:17.665 INFO:teuthology.orchestra.run.smithi184.stdout:(1/33): ceph-18.2.4-820.g48a7de7e.el9.x86_64.rp 50 kB/s | 6.5 kB 00:00 2024-09-13T07:10:17.849 INFO:teuthology.orchestra.run.smithi184.stdout:(2/33): ceph-mds-18.2.4-820.g48a7de7e.el9.x86_6 6.7 MB/s | 2.1 MB 00:00 2024-09-13T07:10:17.891 INFO:teuthology.orchestra.run.smithi184.stdout:(3/33): ceph-mgr-18.2.4-820.g48a7de7e.el9.x86_6 4.1 MB/s | 1.5 MB 00:00 2024-09-13T07:10:17.991 INFO:teuthology.orchestra.run.smithi184.stdout:(4/33): ceph-mgr-modules-core-18.2.4-820.g48a7d 2.4 MB/s | 246 kB 00:00 2024-09-13T07:10:18.058 INFO:teuthology.orchestra.run.smithi184.stdout:(5/33): ceph-mon-18.2.4-820.g48a7de7e.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-13T07:10:18.183 INFO:teuthology.orchestra.run.smithi184.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-13T07:10:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T07:10:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 1/4 2024-09-13T07:10:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2/4 2024-09-13T07:10:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T07:10:18.225 INFO:teuthology.orchestra.run.smithi184.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-13T07:10:18.384 INFO:teuthology.orchestra.run.smithi184.stdout:(8/33): ceph-osd-18.2.4-820.g48a7de7e.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-13T07:10:18.409 INFO:teuthology.orchestra.run.smithi184.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 736 kB/s | 135 kB 00:00 2024-09-13T07:10:18.442 INFO:teuthology.orchestra.run.smithi184.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 3.2 MB/s | 1.2 MB 00:00 2024-09-13T07:10:18.468 INFO:teuthology.orchestra.run.smithi184.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-13T07:10:18.510 INFO:teuthology.orchestra.run.smithi184.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 995 kB/s | 125 kB 00:00 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:18.530 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:18.543 INFO:teuthology.orchestra.run.smithi184.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 462 kB/s | 35 kB 00:00 2024-09-13T07:10:18.602 INFO:teuthology.orchestra.run.smithi184.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-13T07:10:18.619 INFO:teuthology.orchestra.run.smithi184.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-13T07:10:18.636 INFO:teuthology.orchestra.run.smithi184.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 404 kB/s | 51 kB 00:00 2024-09-13T07:10:18.677 INFO:teuthology.orchestra.run.smithi184.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 576 kB/s | 43 kB 00:00 2024-09-13T07:10:18.703 INFO:teuthology.orchestra.run.smithi184.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 457 kB/s | 11 kB 00:00 2024-09-13T07:10:18.728 INFO:teuthology.orchestra.run.smithi184.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-13T07:10:18.753 INFO:teuthology.orchestra.run.smithi184.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-13T07:10:18.779 INFO:teuthology.orchestra.run.smithi184.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-13T07:10:18.804 INFO:teuthology.orchestra.run.smithi184.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 385 kB/s | 19 kB 00:00 2024-09-13T07:10:18.829 INFO:teuthology.orchestra.run.smithi184.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-13T07:10:18.855 INFO:teuthology.orchestra.run.smithi184.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-13T07:10:18.880 INFO:teuthology.orchestra.run.smithi184.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-13T07:10:18.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-09-13T07:10:18.905 INFO:teuthology.orchestra.run.smithi184.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-13T07:10:18.931 INFO:teuthology.orchestra.run.smithi184.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-13T07:10:18.956 INFO:teuthology.orchestra.run.smithi184.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-13T07:10:18.981 INFO:teuthology.orchestra.run.smithi184.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 710 kB/s | 36 kB 00:00 2024-09-13T07:10:19.006 INFO:teuthology.orchestra.run.smithi184.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-13T07:10:19.031 INFO:teuthology.orchestra.run.smithi184.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-13T07:10:19.057 INFO:teuthology.orchestra.run.smithi184.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-09-13T07:10:19.082 INFO:teuthology.orchestra.run.smithi184.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-13T07:10:19.086 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:19.086 INFO:teuthology.orchestra.run.smithi184.stdout:Total 16 MB/s | 29 MB 00:01 2024-09-13T07:10:19.201 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:19.237 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:19.237 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:19.486 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:37 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:19.626 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:19.628 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:10:19.628 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 6.5 k 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 2.1 M 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 1.5 M 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 246 k 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.7 M 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 17 M 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T07:10:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T07:10:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T07:10:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T07:10:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:19.632 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:19.632 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:19.632 INFO:teuthology.orchestra.run.smithi031.stdout:Install 33 Packages 2024-09-13T07:10:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:19.632 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 29 M 2024-09-13T07:10:19.633 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 104 M 2024-09-13T07:10:19.633 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:19.652 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:19.652 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:10:20.049 INFO:teuthology.orchestra.run.smithi031.stdout:(1/33): ceph-18.2.4-820.g48a7de7e.el9.x86_64.rp 50 kB/s | 6.5 kB 00:00 2024-09-13T07:10:20.224 INFO:teuthology.orchestra.run.smithi031.stdout:(2/33): ceph-mds-18.2.4-820.g48a7de7e.el9.x86_6 6.9 MB/s | 2.1 MB 00:00 2024-09-13T07:10:20.266 INFO:teuthology.orchestra.run.smithi031.stdout:(3/33): ceph-mgr-18.2.4-820.g48a7de7e.el9.x86_6 4.2 MB/s | 1.5 MB 00:00 2024-09-13T07:10:20.366 INFO:teuthology.orchestra.run.smithi031.stdout:(4/33): ceph-mgr-modules-core-18.2.4-820.g48a7d 2.4 MB/s | 246 kB 00:00 2024-09-13T07:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout:(5/33): ceph-mon-18.2.4-820.g48a7de7e.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-13T07:10:20.550 INFO:teuthology.orchestra.run.smithi031.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-13T07:10:20.592 INFO:teuthology.orchestra.run.smithi031.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-13T07:10:20.634 INFO:teuthology.orchestra.run.smithi031.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-13T07:10:20.801 INFO:teuthology.orchestra.run.smithi031.stdout:(9/33): ceph-osd-18.2.4-820.g48a7de7e.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-13T07:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 650 kB/s | 125 kB 00:00 2024-09-13T07:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 3.0 MB/s | 1.2 MB 00:00 2024-09-13T07:10:20.861 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:10:20.918 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T07:10:20.927 INFO:teuthology.orchestra.run.smithi031.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 523 kB/s | 35 kB 00:00 2024-09-13T07:10:20.950 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T07:10:20.952 INFO:teuthology.orchestra.run.smithi031.stdout:(13/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.4 MB/s | 215 kB 00:00 2024-09-13T07:10:20.977 INFO:teuthology.orchestra.run.smithi031.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-13T07:10:20.983 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T07:10:21.003 INFO:teuthology.orchestra.run.smithi031.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-13T07:10:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 651 kB/s | 43 kB 00:00 2024-09-13T07:10:21.070 INFO:teuthology.orchestra.run.smithi031.stdout:(17/33): python3-toml-0.10.2-6.el9.noarch.rpm 354 kB/s | 42 kB 00:00 2024-09-13T07:10:21.076 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T07:10:21.103 INFO:teuthology.orchestra.run.smithi031.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-13T07:10:21.129 INFO:teuthology.orchestra.run.smithi031.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-09-13T07:10:21.154 INFO:teuthology.orchestra.run.smithi031.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 367 kB/s | 18 kB 00:00 2024-09-13T07:10:21.162 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T07:10:21.187 INFO:teuthology.orchestra.run.smithi031.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.5 MB/s | 358 kB 00:00 2024-09-13T07:10:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 332 kB/s | 19 kB 00:00 2024-09-13T07:10:21.214 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T07:10:21.238 INFO:teuthology.orchestra.run.smithi031.stdout:(23/33): python3-jaraco-collections-3.0.0-8.el9 276 kB/s | 23 kB 00:00 2024-09-13T07:10:21.249 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T07:10:21.263 INFO:teuthology.orchestra.run.smithi031.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-13T07:10:21.281 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T07:10:21.288 INFO:teuthology.orchestra.run.smithi031.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 613 kB/s | 46 kB 00:00 2024-09-13T07:10:21.314 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T07:10:21.322 INFO:teuthology.orchestra.run.smithi031.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 941 kB/s | 79 kB 00:00 2024-09-13T07:10:21.347 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T07:10:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 281 kB/s | 16 kB 00:00 2024-09-13T07:10:21.372 INFO:teuthology.orchestra.run.smithi031.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-13T07:10:21.387 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T07:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-13T07:10:21.422 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T07:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-13T07:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-13T07:10:21.456 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T07:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-13T07:10:21.490 INFO:teuthology.orchestra.run.smithi031.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-13T07:10:21.494 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:21.495 INFO:teuthology.orchestra.run.smithi031.stdout:Total 16 MB/s | 29 MB 00:01 2024-09-13T07:10:21.509 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T07:10:21.585 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T07:10:21.617 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:10:21.658 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:10:21.659 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:10:21.733 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T07:10:21.820 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T07:10:21.873 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T07:10:21.911 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T07:10:21.961 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T07:10:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T07:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:10:22.215 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T07:10:22.274 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T07:10:22.354 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T07:10:22.451 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T07:10:22.513 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T07:10:22.633 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T07:10:22.800 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 28/33 2024-09-13T07:10:22.826 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T07:10:22.853 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T07:10:22.853 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:22.853 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T07:10:22.853 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T07:10:22.853 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T07:10:22.853 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:10:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T07:10:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T07:10:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T07:10:23.591 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T07:10:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T07:10:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T07:10:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T07:10:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T07:10:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T07:10:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T07:10:23.913 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T07:10:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T07:10:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T07:10:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T07:10:24.139 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T07:10:24.285 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T07:10:24.290 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T07:10:24.313 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T07:10:24.313 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:24.313 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T07:10:24.313 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T07:10:24.313 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T07:10:24.313 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T07:10:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T07:10:24.479 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T07:10:24.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T07:10:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T07:10:24.752 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T07:10:24.777 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T07:10:24.777 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:24.778 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T07:10:24.778 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T07:10:24.778 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T07:10:24.778 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:24.798 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T07:10:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T07:10:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T07:10:24.978 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T07:10:25.002 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T07:10:25.002 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:25.002 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T07:10:25.002 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T07:10:25.002 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T07:10:25.002 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:25.037 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T07:10:25.066 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T07:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T07:10:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T07:10:25.392 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 28/33 2024-09-13T07:10:25.418 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T07:10:25.445 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T07:10:25.445 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:25.446 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T07:10:25.446 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T07:10:25.446 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T07:10:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:26.847 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 1/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 3/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 4/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 5/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 6/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T07:10:26.848 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T07:10:26.849 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T07:10:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T07:10:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T07:10:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T07:10:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T07:10:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T07:10:26.850 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T07:10:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T07:10:26.851 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T07:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T07:10:26.997 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T07:10:26.997 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:26.997 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T07:10:26.997 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T07:10:26.997 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T07:10:26.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T07:10:27.510 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T07:10:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T07:10:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T07:10:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T07:10:27.511 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:27.727 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T07:10:27.730 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T07:10:27.730 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:27.730 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:27.730 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:27.730 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T07:10:27.731 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T07:10:27.732 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T07:10:27.733 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T07:10:27.733 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T07:10:27.733 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T07:10:27.733 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:27.733 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:27.752 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T07:10:27.753 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:27.753 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T07:10:27.753 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T07:10:27.753 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T07:10:27.753 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:27.818 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T07:10:28.108 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-base 2024-09-13T07:10:28.680 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:47 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:28.769 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:10:28.811 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:28.812 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:10:28.812 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:28.866 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install cephadm 2024-09-13T07:10:29.424 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 1/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 3/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 4/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 5/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 6/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T07:10:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T07:10:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T07:10:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T07:10:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T07:10:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T07:10:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T07:10:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T07:10:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T07:10:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T07:10:29.555 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:29.555 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 226 k 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 226 k 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 217 k 2024-09-13T07:10:29.556 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:29.770 INFO:teuthology.orchestra.run.smithi184.stdout:cephadm-18.2.4-820.g48a7de7e.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-13T07:10:29.770 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:29.770 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-13T07:10:29.771 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:29.776 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:29.776 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:29.779 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:29.779 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:10:29.847 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:10:30.274 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:30.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T07:10:30.354 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:30.354 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T07:10:30.355 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T07:10:30.356 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:30.690 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:30.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-09-13T07:10:30.937 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:30.937 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:30.937 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:30.937 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:30.937 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:30.937 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:31.157 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T07:10:31.295 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:31.388 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:10:31.433 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:31.434 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:10:31.434 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:31.494 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-09-13T07:10:31.730 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:31.865 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:31.866 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:31.866 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-13T07:10:31.866 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:31.866 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:31.866 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 143 k 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 143 k 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 395 k 2024-09-13T07:10:31.867 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:32.067 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-immutable-object-cache-18.2.4-820.g48a7de7 714 kB/s | 143 kB 00:00 2024-09-13T07:10:32.067 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:32.068 INFO:teuthology.orchestra.run.smithi184.stdout:Total 709 kB/s | 143 kB 00:00 2024-09-13T07:10:32.068 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:32.075 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:32.075 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:32.081 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:32.133 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:32.134 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:10:32.218 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:32.219 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:10:32.219 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:32.219 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 226 k 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 226 k 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 217 k 2024-09-13T07:10:32.220 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:32.276 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:10:32.343 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T07:10:32.368 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T07:10:32.369 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:32.369 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T07:10:32.369 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:32.450 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-18.2.4-820.g48a7de7e.el9.noarch.rpm 986 kB/s | 226 kB 00:00 2024-09-13T07:10:32.450 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:32.450 INFO:teuthology.orchestra.run.smithi031.stdout:Total 979 kB/s | 226 kB 00:00 2024-09-13T07:10:32.450 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:10:32.456 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:10:32.456 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:10:32.459 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:10:32.459 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:10:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:10:32.946 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:33.323 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T07:10:33.323 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:33.323 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:33.323 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:33.323 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:33.323 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:33.375 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:33.528 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr 2024-09-13T07:10:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T07:10:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:33.636 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:33.636 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:33.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T07:10:34.107 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:34.197 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:10:34.240 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:34.241 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:10:34.241 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:34.300 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T07:10:34.439 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:34.576 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:34.577 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-13T07:10:34.577 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:34.577 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 143 k 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 143 k 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 395 k 2024-09-13T07:10:34.578 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:34.788 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-18.2.4-820.g48a7de7 678 kB/s | 143 kB 00:00 2024-09-13T07:10:34.789 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:34.789 INFO:teuthology.orchestra.run.smithi031.stdout:Total 673 kB/s | 143 kB 00:00 2024-09-13T07:10:34.789 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:10:34.797 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:10:34.798 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:10:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:10:34.859 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:10:34.865 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:34.998 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:34.998 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 3.5 M 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 24 k 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 16 k 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:34.999 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:35.000 INFO:teuthology.orchestra.run.smithi184.stdout:Install 5 Packages 2024-09-13T07:10:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:35.000 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.8 M 2024-09-13T07:10:35.000 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 87 M 2024-09-13T07:10:35.000 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:35.005 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:10:35.073 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T07:10:35.099 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T07:10:35.099 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:10:35.099 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T07:10:35.099 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:35.220 INFO:teuthology.orchestra.run.smithi184.stdout:(1/5): ceph-prometheus-alerts-18.2.4-820.g48a7d 98 kB/s | 16 kB 00:00 2024-09-13T07:10:35.245 INFO:teuthology.orchestra.run.smithi184.stdout:(2/5): ceph-grafana-dashboards-18.2.4-820.g48a7 126 kB/s | 24 kB 00:00 2024-09-13T07:10:35.295 INFO:teuthology.orchestra.run.smithi184.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 408 kB/s | 31 kB 00:00 2024-09-13T07:10:35.321 INFO:teuthology.orchestra.run.smithi184.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-13T07:10:35.429 INFO:teuthology.orchestra.run.smithi184.stdout:(5/5): ceph-mgr-dashboard-18.2.4-820.g48a7de7e. 9.4 MB/s | 3.5 MB 00:00 2024-09-13T07:10:35.433 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:35.433 INFO:teuthology.orchestra.run.smithi184.stdout:Total 8.7 MB/s | 3.8 MB 00:00 2024-09-13T07:10:35.433 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:35.525 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:35.525 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:10:35.804 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:10:35.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-13T07:10:35.926 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-13T07:10:35.954 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T07:10:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T07:10:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:36.044 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:10:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:36.044 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:36.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-09-13T07:10:36.873 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:36.966 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:10:37.009 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 4/5 2024-09-13T07:10:37.011 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:37.012 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:10:37.013 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:37.050 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T07:10:37.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T07:10:37.663 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:37.803 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:37.803 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 3.5 M 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 24 k 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 16 k 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:37.804 INFO:teuthology.orchestra.run.smithi031.stdout:Install 5 Packages 2024-09-13T07:10:37.805 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:37.805 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.8 M 2024-09-13T07:10:37.805 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 87 M 2024-09-13T07:10:37.805 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:38.125 INFO:teuthology.orchestra.run.smithi031.stdout:(1/5): ceph-prometheus-alerts-18.2.4-820.g48a7d 99 kB/s | 16 kB 00:00 2024-09-13T07:10:38.150 INFO:teuthology.orchestra.run.smithi031.stdout:(2/5): ceph-grafana-dashboards-18.2.4-820.g48a7 127 kB/s | 24 kB 00:00 2024-09-13T07:10:38.192 INFO:teuthology.orchestra.run.smithi031.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-13T07:10:38.193 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T07:10:38.193 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 1/5 2024-09-13T07:10:38.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2/5 2024-09-13T07:10:38.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T07:10:38.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-13T07:10:38.217 INFO:teuthology.orchestra.run.smithi031.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-13T07:10:38.334 INFO:teuthology.orchestra.run.smithi031.stdout:(5/5): ceph-mgr-dashboard-18.2.4-820.g48a7de7e. 9.4 MB/s | 3.5 MB 00:00 2024-09-13T07:10:38.338 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:38.338 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.1 MB/s | 3.8 MB 00:00 2024-09-13T07:10:38.338 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:10:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:10:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:10:38.446 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:10:38.447 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:10:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-13T07:10:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:38.452 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T07:10:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T07:10:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:38.453 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:38.664 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T07:10:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:10:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-13T07:10:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-13T07:10:38.851 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T07:10:39.233 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:58 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:39.365 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:39.366 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-13T07:10:39.366 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:10:39.366 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-13T07:10:39.366 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 7.4 M 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T07:10:39.367 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:39.368 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:39.368 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-09-13T07:10:39.368 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-09-13T07:10:39.368 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:39.368 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 38 M 2024-09-13T07:10:39.368 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 212 M 2024-09-13T07:10:39.369 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:39.961 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.4-82 17 MB/s | 7.4 MB 00:00 2024-09-13T07:10:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 4/5 2024-09-13T07:10:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T07:10:40.386 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 221 kB/s | 191 kB 00:00 2024-09-13T07:10:40.511 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-13T07:10:41.003 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 37 kB/s | 18 kB 00:00 2024-09-13T07:10:41.062 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): libgfortran-11.5.0-2.el9.x86_64.rpm 520 kB/s | 801 kB 00:01 2024-09-13T07:10:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T07:10:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 1/5 2024-09-13T07:10:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2/5 2024-09-13T07:10:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T07:10:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-13T07:10:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-13T07:10:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:41.401 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T07:10:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T07:10:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:41.402 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:41.571 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-13T07:10:41.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T07:10:41.629 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-09-13T07:10:42.171 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.5 MB/s | 4.9 MB 00:01 2024-09-13T07:10:42.210 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:00 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:42.330 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 323 kB/s | 245 kB 00:00 2024-09-13T07:10:42.350 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:42.351 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-13T07:10:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:10:42.351 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-13T07:10:42.351 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 7.4 M 2024-09-13T07:10:42.351 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:42.352 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:42.353 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-13T07:10:42.353 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-13T07:10:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:42.353 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-09-13T07:10:42.353 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 212 M 2024-09-13T07:10:42.353 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:42.593 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-13T07:10:42.633 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-13T07:10:42.716 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 266 kB/s | 33 kB 00:00 2024-09-13T07:10:42.780 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-09-13T07:10:42.783 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 273 kB/s | 18 kB 00:00 2024-09-13T07:10:42.859 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-09-13T07:10:42.884 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-13T07:10:42.947 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 323 kB/s | 250 kB 00:00 2024-09-13T07:10:42.993 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): ceph-mgr-diskprediction-local-18.2.4-82 13 MB/s | 7.4 MB 00:00 2024-09-13T07:10:43.026 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-13T07:10:43.118 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 19 MB/s | 4.9 MB 00:00 2024-09-13T07:10:43.143 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-13T07:10:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 57 MB/s | 16 MB 00:00 2024-09-13T07:10:43.619 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.1 MB/s | 5.1 MB 00:00 2024-09-13T07:10:43.619 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:43.620 INFO:teuthology.orchestra.run.smithi031.stdout:Total 30 MB/s | 38 MB 00:01 2024-09-13T07:10:43.914 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-13T07:10:43.915 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:43.915 INFO:teuthology.orchestra.run.smithi184.stdout:Total 8.4 MB/s | 38 MB 00:04 2024-09-13T07:10:43.986 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:10:44.013 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:10:44.013 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:10:44.276 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:44.303 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:44.303 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:44.552 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:10:44.553 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:10:44.829 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:44.829 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:10:45.343 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:10:45.417 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T07:10:45.464 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T07:10:45.564 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T07:10:45.595 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:10:45.602 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T07:10:45.680 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T07:10:45.729 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T07:10:45.795 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T07:10:45.830 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T07:10:45.838 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T07:10:45.880 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T07:10:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T07:10:46.067 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T07:10:46.103 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T07:10:46.181 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T07:10:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T07:10:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T07:10:46.831 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T07:10:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T07:10:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T07:10:48.201 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T07:10:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T07:10:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T07:10:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T07:10:49.252 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T07:10:51.755 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T07:10:51.755 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 1/12 2024-09-13T07:10:51.755 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T07:10:51.755 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T07:10:51.755 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T07:10:51.755 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T07:10:51.755 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T07:10:51.756 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T07:10:51.756 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T07:10:51.756 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T07:10:51.756 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T07:10:51.756 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T07:10:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T07:10:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:52.183 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T07:10:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:52.185 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:52.253 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T07:10:52.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 1/12 2024-09-13T07:10:52.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T07:10:52.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T07:10:52.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T07:10:52.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T07:10:52.258 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T07:10:52.258 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T07:10:52.258 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T07:10:52.258 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T07:10:52.258 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T07:10:52.258 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T07:10:52.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T07:10:52.721 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T07:10:52.722 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T07:10:52.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T07:10:52.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T07:10:52.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T07:10:52.722 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T07:10:52.722 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:52.722 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:53.123 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-rook 2024-09-13T07:10:53.141 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:11 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:53.284 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:53.285 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:10:53.285 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:53.285 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 49 k 2024-09-13T07:10:53.285 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T07:10:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T07:10:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:53.287 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:53.287 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:53.287 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-09-13T07:10:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:53.288 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-09-13T07:10:53.288 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-09-13T07:10:53.288 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:53.719 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:12 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:53.840 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 276 kB/s | 26 kB 00:00 2024-09-13T07:10:53.862 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:53.863 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:53.863 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 49 k 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T07:10:53.864 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout:Install 13 Packages 2024-09-13T07:10:53.865 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:53.866 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 162 kB/s | 19 kB 00:00 2024-09-13T07:10:53.866 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 2.2 M 2024-09-13T07:10:53.866 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 26 M 2024-09-13T07:10:53.866 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:53.923 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): ceph-mgr-rook-18.2.4-820.g48a7de7e.el9. 279 kB/s | 49 kB 00:00 2024-09-13T07:10:53.973 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-13T07:10:53.999 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-13T07:10:54.041 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 801 kB/s | 54 kB 00:00 2024-09-13T07:10:54.108 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 296 kB/s | 32 kB 00:00 2024-09-13T07:10:54.127 INFO:teuthology.orchestra.run.smithi184.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 242 kB/s | 19 kB 00:00 2024-09-13T07:10:54.141 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 146 kB/s | 14 kB 00:00 2024-09-13T07:10:54.161 INFO:teuthology.orchestra.run.smithi184.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 227 kB/s | 26 kB 00:00 2024-09-13T07:10:54.167 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-13T07:10:54.209 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.2 MB/s | 220 kB 00:00 2024-09-13T07:10:54.228 INFO:teuthology.orchestra.run.smithi184.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-13T07:10:54.234 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-13T07:10:54.254 INFO:teuthology.orchestra.run.smithi184.stdout:(4/13): ceph-mgr-rook-18.2.4-820.g48a7de7e.el9. 240 kB/s | 49 kB 00:00 2024-09-13T07:10:54.259 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-09-13T07:10:54.279 INFO:teuthology.orchestra.run.smithi184.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-13T07:10:54.292 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-13T07:10:54.295 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:54.295 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.2 MB/s | 2.2 MB 00:01 2024-09-13T07:10:54.304 INFO:teuthology.orchestra.run.smithi184.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-09-13T07:10:54.346 INFO:teuthology.orchestra.run.smithi184.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-13T07:10:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:10:54.371 INFO:teuthology.orchestra.run.smithi184.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 456 kB/s | 54 kB 00:00 2024-09-13T07:10:54.377 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:10:54.378 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:10:54.396 INFO:teuthology.orchestra.run.smithi184.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-13T07:10:54.430 INFO:teuthology.orchestra.run.smithi184.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-13T07:10:54.456 INFO:teuthology.orchestra.run.smithi184.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-13T07:10:54.474 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:10:54.474 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:10:54.481 INFO:teuthology.orchestra.run.smithi184.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-13T07:10:54.531 INFO:teuthology.orchestra.run.smithi184.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-13T07:10:54.534 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:54.534 INFO:teuthology.orchestra.run.smithi184.stdout:Total 3.3 MB/s | 2.2 MB 00:00 2024-09-13T07:10:54.598 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:54.614 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:54.614 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:54.702 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:54.703 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:10:54.995 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:10:55.182 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T07:10:55.215 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:10:55.237 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T07:10:55.302 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T07:10:55.337 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T07:10:55.379 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T07:10:55.379 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T07:10:55.432 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T07:10:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T07:10:55.487 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T07:10:55.533 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T07:10:55.575 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T07:10:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T07:10:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T07:10:55.675 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T07:10:55.779 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T07:10:55.827 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T07:10:56.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T07:10:56.485 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T07:10:56.519 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T07:10:56.535 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T07:10:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T07:10:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T07:10:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T07:10:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T07:10:56.672 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T07:10:56.702 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 1/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T07:10:57.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T07:10:57.403 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T07:10:57.403 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T07:10:57.403 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T07:10:57.403 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T07:10:57.403 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T07:10:57.522 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T07:10:57.522 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 1/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T07:10:57.523 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T07:10:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T07:10:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T07:10:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T07:10:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T07:10:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T07:10:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T07:10:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T07:10:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:57.734 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:10:57.844 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T07:10:57.845 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T07:10:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T07:10:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T07:10:57.846 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:57.846 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:10:58.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T07:10:58.112 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T07:10:58.633 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:16 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:10:58.686 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:17 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:10:58.772 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 139 k 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T07:10:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:10:58.774 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-09-13T07:10:58.775 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-09-13T07:10:58.775 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:10:58.818 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:10:58.819 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:10:58.819 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:58.819 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:10:58.819 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 139 k 2024-09-13T07:10:58.819 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout:Install 6 Packages 2024-09-13T07:10:58.820 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:10:58.821 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 7.0 M 2024-09-13T07:10:58.821 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 33 M 2024-09-13T07:10:58.821 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:10:59.078 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): ceph-mgr-cephadm-18.2.4-820.g48a7de7e.el 601 kB/s | 139 kB 00:00 2024-09-13T07:10:59.137 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-13T07:10:59.153 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-13T07:10:59.179 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-13T07:10:59.416 INFO:teuthology.orchestra.run.smithi184.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-13T07:10:59.466 INFO:teuthology.orchestra.run.smithi184.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-13T07:10:59.479 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 393 kB/s | 248 kB 00:00 2024-09-13T07:10:59.492 INFO:teuthology.orchestra.run.smithi184.stdout:(3/6): ceph-mgr-cephadm-18.2.4-820.g48a7de7e.el 602 kB/s | 139 kB 00:00 2024-09-13T07:10:59.517 INFO:teuthology.orchestra.run.smithi184.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-13T07:10:59.559 INFO:teuthology.orchestra.run.smithi184.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-13T07:10:59.634 INFO:teuthology.orchestra.run.smithi184.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-13T07:10:59.637 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:10:59.637 INFO:teuthology.orchestra.run.smithi184.stdout:Total 8.6 MB/s | 7.0 MB 00:00 2024-09-13T07:10:59.709 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:10:59.720 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:10:59.720 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:10:59.818 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:10:59.818 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:00.119 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:00.290 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T07:11:00.345 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T07:11:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T07:11:00.896 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.9 MB/s | 6.0 MB 00:02 2024-09-13T07:11:00.898 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:00.898 INFO:teuthology.orchestra.run.smithi031.stdout:Total 3.3 MB/s | 7.0 MB 00:02 2024-09-13T07:11:00.947 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T07:11:00.974 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:00.986 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:00.987 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:01.009 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T07:11:01.032 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T07:11:01.093 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:01.093 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:01.419 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T07:11:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T07:11:01.939 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T07:11:01.939 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/6 2024-09-13T07:11:01.939 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T07:11:01.939 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T07:11:01.939 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T07:11:01.939 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T07:11:02.197 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T07:11:02.197 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:02.197 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:02.197 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:11:02.197 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T07:11:02.198 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T07:11:02.198 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T07:11:02.198 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T07:11:02.198 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T07:11:02.198 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:02.198 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T07:11:02.329 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T07:11:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T07:11:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T07:11:02.465 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-fuse 2024-09-13T07:11:03.036 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:22 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 852 k 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-09-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:03.173 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 932 k 2024-09-13T07:11:03.173 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 2.7 M 2024-09-13T07:11:03.173 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:11:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T07:11:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/6 2024-09-13T07:11:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T07:11:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T07:11:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T07:11:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T07:11:03.475 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T07:11:03.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T07:11:03.477 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:03.477 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:03.486 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): ceph-fuse-18.2.4-820.g48a7de7e.el9.x86_6 3.2 MB/s | 852 kB 00:00 2024-09-13T07:11:03.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-09-13T07:11:03.862 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 125 kB/s | 80 kB 00:00 2024-09-13T07:11:03.862 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:03.863 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.3 MB/s | 932 kB 00:00 2024-09-13T07:11:03.875 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:11:03.897 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:11:03.897 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:11:03.982 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:11:03.982 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:04.187 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:04.325 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:22 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T07:11:04.410 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T07:11:04.463 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:04.464 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:04.464 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:11:04.464 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:04.464 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 852 k 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 932 k 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.7 M 2024-09-13T07:11:04.465 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:11:04.614 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 828 kB/s | 80 kB 00:00 2024-09-13T07:11:04.789 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-fuse-18.2.4-820.g48a7de7e.el9.x86_6 3.1 MB/s | 852 kB 00:00 2024-09-13T07:11:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.8 MB/s | 932 kB 00:00 2024-09-13T07:11:04.802 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:04.825 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:04.825 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:04.909 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:04.910 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:05.143 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T07:11:05.143 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/2 2024-09-13T07:11:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T07:11:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T07:11:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T07:11:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:05.415 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T07:11:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:05.415 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:05.724 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-volume 2024-09-13T07:11:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T07:11:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/2 2024-09-13T07:11:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T07:11:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:06.249 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T07:11:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:06.249 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:06.309 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:25 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:06.445 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 264 k 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:06.446 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-09-13T07:11:06.447 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:06.447 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 342 k 2024-09-13T07:11:06.447 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 1.5 M 2024-09-13T07:11:06.447 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:11:06.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-volume 2024-09-13T07:11:06.633 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 557 kB/s | 77 kB 00:00 2024-09-13T07:11:06.725 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-volume-18.2.4-820.g48a7de7e.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-13T07:11:06.725 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:06.726 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.2 MB/s | 342 kB 00:00 2024-09-13T07:11:06.738 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:11:06.748 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:11:06.748 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:11:06.783 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:11:06.783 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:06.931 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:07.117 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-13T07:11:07.140 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:25 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:07.170 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T07:11:07.186 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T07:11:07.186 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:11:07.186 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T07:11:07.186 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:07.279 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:07.279 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-13T07:11:07.279 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 264 k 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-09-13T07:11:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:07.281 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 342 k 2024-09-13T07:11:07.281 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.5 M 2024-09-13T07:11:07.281 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:11:07.693 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 495 kB/s | 77 kB 00:00 2024-09-13T07:11:07.760 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-volume-18.2.4-820.g48a7de7e.el9.noa 1.2 MB/s | 264 kB 00:00 2024-09-13T07:11:07.760 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:07.760 INFO:teuthology.orchestra.run.smithi031.stdout:Total 711 kB/s | 342 kB 00:00 2024-09-13T07:11:07.773 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:07.784 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:07.784 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:07.819 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:07.820 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:07.904 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 1/2 2024-09-13T07:11:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:08.152 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-13T07:11:08.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T07:11:08.204 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-13T07:11:08.204 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:08.204 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:08.204 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:11:08.204 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T07:11:08.204 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:08.204 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:08.205 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T07:11:08.205 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:11:08.205 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T07:11:08.205 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:08.507 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados-devel 2024-09-13T07:11:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 1/2 2024-09-13T07:11:09.087 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:28 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:09.222 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-13T07:11:09.222 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:09.222 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:09.222 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T07:11:09.222 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T07:11:09.222 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:09.222 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:09.223 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:09.223 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 127 k 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 127 k 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 456 k 2024-09-13T07:11:09.224 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:11:09.431 INFO:teuthology.orchestra.run.smithi184.stdout:librados-devel-18.2.4-820.g48a7de7e.el9.x86_64. 613 kB/s | 127 kB 00:00 2024-09-13T07:11:09.432 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:09.432 INFO:teuthology.orchestra.run.smithi184.stdout:Total 608 kB/s | 127 kB 00:00 2024-09-13T07:11:09.432 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:11:09.437 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:11:09.437 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:11:09.500 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:11:09.500 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:09.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-09-13T07:11:09.655 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:09.761 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:28 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:10.231 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 127 k 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-13T07:11:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-09-13T07:11:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 456 k 2024-09-13T07:11:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:11:10.473 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-18.2.4-820.g48a7de7e.el9.x86_64. 614 kB/s | 127 kB 00:00 2024-09-13T07:11:10.474 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:10.474 INFO:teuthology.orchestra.run.smithi031.stdout:Total 609 kB/s | 127 kB 00:00 2024-09-13T07:11:10.474 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:10.479 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:10.480 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:10.490 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:10.490 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:10.490 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:10.490 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:10.490 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:10.490 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:10.546 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:10.547 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:10.766 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs2 2024-09-13T07:11:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:11.345 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:30 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:11.439 INFO:teuthology.orchestra.run.smithi184.stdout:Package libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:11.485 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:11.486 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:11:11.486 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:11.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:11.505 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:11.505 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:11.505 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:11.505 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:11.505 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:11.588 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs-devel 2024-09-13T07:11:11.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-09-13T07:11:12.150 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:31 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:12.282 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:12.283 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:12.283 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 31 k 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 31 k 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 139 k 2024-09-13T07:11:12.284 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:11:12.342 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:30 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:12.435 INFO:teuthology.orchestra.run.smithi184.stdout:libcephfs-devel-18.2.4-820.g48a7de7e.el9.x86_64 207 kB/s | 31 kB 00:00 2024-09-13T07:11:12.435 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:12.435 INFO:teuthology.orchestra.run.smithi184.stdout:Total 205 kB/s | 31 kB 00:00 2024-09-13T07:11:12.435 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:11:12.436 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:12.438 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:11:12.438 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:11:12.452 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:11:12.452 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:12.481 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:12.482 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:11:12.482 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:12.533 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:12.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-09-13T07:11:12.639 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:13.005 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:31 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:13.229 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:13.229 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:13.229 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:13.229 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:13.229 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:13.229 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:13.294 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 31 k 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:13.295 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 31 k 2024-09-13T07:11:13.296 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 139 k 2024-09-13T07:11:13.296 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:11:13.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados2 2024-09-13T07:11:13.459 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-18.2.4-820.g48a7de7e.el9.x86_64 190 kB/s | 31 kB 00:00 2024-09-13T07:11:13.459 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:13.460 INFO:teuthology.orchestra.run.smithi031.stdout:Total 189 kB/s | 31 kB 00:00 2024-09-13T07:11:13.460 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:13.464 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:13.464 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:13.478 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:13.478 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:13.992 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:32 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:14.083 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:14.125 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:14.126 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:11:14.126 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:14.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librbd1 2024-09-13T07:11:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:14.304 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:14.304 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:14.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-09-13T07:11:14.754 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:14.843 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:14.885 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:14.886 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:11:14.887 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:14.958 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rados 2024-09-13T07:11:15.083 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:15.177 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:15.221 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:15.223 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:11:15.223 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:15.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-09-13T07:11:15.520 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:34 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:15.609 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:15.652 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:15.653 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:11:15.653 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:15.728 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rgw 2024-09-13T07:11:15.902 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:15.996 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:16.041 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:16.042 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:11:16.042 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:16.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-09-13T07:11:16.291 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:16.380 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:16.423 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:16.424 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:11:16.424 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:16.490 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-cephfs 2024-09-13T07:11:16.714 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:34 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:16.808 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:16.854 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:16.855 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:11:16.855 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:16.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-09-13T07:11:17.058 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:36 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:17.148 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:17.192 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:17.194 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:11:17.194 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:17.268 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rbd 2024-09-13T07:11:17.521 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:17.617 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:17.664 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:17.665 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:11:17.665 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:17.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-09-13T07:11:17.832 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:36 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:17.922 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:17.964 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:17.965 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-09-13T07:11:17.965 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:18.041 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-fuse 2024-09-13T07:11:18.325 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:36 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:18.418 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:18.463 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:18.464 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:11:18.465 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:18.605 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:37 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:18.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-09-13T07:11:18.738 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 87 k 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-13T07:11:18.739 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:18.740 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 87 k 2024-09-13T07:11:18.740 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 227 k 2024-09-13T07:11:18.740 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:11:18.912 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-fuse-18.2.4-820.g48a7de7e.el9.x86_64.rpm 506 kB/s | 87 kB 00:00 2024-09-13T07:11:18.912 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:18.913 INFO:teuthology.orchestra.run.smithi184.stdout:Total 502 kB/s | 87 kB 00:00 2024-09-13T07:11:18.913 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:11:18.919 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:11:18.920 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:11:18.982 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:11:18.982 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:19.121 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:37 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:19.225 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:19.316 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T07:11:19.362 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:19.363 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-13T07:11:19.363 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:19.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-09-13T07:11:19.631 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:19.863 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:19.863 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:19.863 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:19.863 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:19.863 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:19.863 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:20.165 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-mirror 2024-09-13T07:11:20.251 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:20.252 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:20.252 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-13T07:11:20.252 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:20.252 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:11:20.252 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 87 k 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 87 k 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 227 k 2024-09-13T07:11:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:11:20.446 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-18.2.4-820.g48a7de7e.el9.x86_64.rpm 452 kB/s | 87 kB 00:00 2024-09-13T07:11:20.446 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:20.446 INFO:teuthology.orchestra.run.smithi031.stdout:Total 449 kB/s | 87 kB 00:00 2024-09-13T07:11:20.447 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:20.454 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:20.454 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:20.522 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:20.522 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:20.733 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:39 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-09-13T07:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:11:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T07:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-13T07:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.0 M 2024-09-13T07:11:20.870 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 12 M 2024-09-13T07:11:20.871 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:11:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:21.182 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-mirror-18.2.4-820.g48a7de7e.el9.x86_64.rpm 9.6 MB/s | 3.0 MB 00:00 2024-09-13T07:11:21.183 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:21.183 INFO:teuthology.orchestra.run.smithi184.stdout:Total 9.5 MB/s | 3.0 MB 00:00 2024-09-13T07:11:21.183 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:11:21.192 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:11:21.192 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:11:21.226 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:21.271 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:11:21.271 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:21.488 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:21.488 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:21.488 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:21.488 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:21.488 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:21.489 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:21.684 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:21.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-09-13T07:11:21.749 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:21.773 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:21.773 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:11:21.774 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T07:11:21.774 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T07:11:21.774 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T07:11:21.774 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:22.345 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:40 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:22.488 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:22.488 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-13T07:11:22.488 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:22.488 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:11:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T07:11:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:22.489 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:11:22.489 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:22.489 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-13T07:11:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:22.489 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.0 M 2024-09-13T07:11:22.489 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 12 M 2024-09-13T07:11:22.489 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:11:22.824 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-18.2.4-820.g48a7de7e.el9.x86_64.rpm 8.9 MB/s | 3.0 MB 00:00 2024-09-13T07:11:22.824 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:22.825 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.9 MB/s | 3.0 MB 00:00 2024-09-13T07:11:22.825 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:22.835 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:22.835 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:22.841 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:22.841 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:22.841 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:22.841 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:22.842 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:22.842 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:22.919 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:22.920 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:23.081 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-nbd 2024-09-13T07:11:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:23.444 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T07:11:23.444 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T07:11:23.444 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T07:11:23.444 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T07:11:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:23.658 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:42 ago on Fri 13 Sep 2024 07:09:41 AM UTC. 2024-09-13T07:11:23.791 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 172 k 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:23.792 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 172 k 2024-09-13T07:11:23.793 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 494 k 2024-09-13T07:11:23.793 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-09-13T07:11:24.002 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-nbd-18.2.4-820.g48a7de7e.el9.x86_64.rpm 822 kB/s | 172 kB 00:00 2024-09-13T07:11:24.003 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:24.003 INFO:teuthology.orchestra.run.smithi184.stdout:Total 815 kB/s | 172 kB 00:00 2024-09-13T07:11:24.003 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-09-13T07:11:24.011 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-09-13T07:11:24.011 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-09-13T07:11:24.074 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-09-13T07:11:24.074 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-09-13T07:11:24.221 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-09-13T07:11:24.324 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:24.626 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:24.626 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:24.763 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:24.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-09-13T07:11:25.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:25.027 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:25.027 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-09-13T07:11:25.027 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:25.027 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:25.027 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-09-13T07:11:25.302 DEBUG:teuthology.parallel:result is None 2024-09-13T07:11:25.472 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 07:09:42 AM UTC. 2024-09-13T07:11:25.613 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-13T07:11:25.613 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 172 k 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 172 k 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 494 k 2024-09-13T07:11:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-13T07:11:25.803 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-18.2.4-820.g48a7de7e.el9.x86_64.rpm 915 kB/s | 172 kB 00:00 2024-09-13T07:11:25.803 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-13T07:11:25.803 INFO:teuthology.orchestra.run.smithi031.stdout:Total 907 kB/s | 172 kB 00:00 2024-09-13T07:11:25.803 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-13T07:11:25.812 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-13T07:11:25.812 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-13T07:11:25.880 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-13T07:11:25.880 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-13T07:11:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-13T07:11:26.145 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T07:11:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:26.850 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-13T07:11:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T07:11:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:26.850 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-13T07:11:27.100 DEBUG:teuthology.parallel:result is None 2024-09-13T07:11:27.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T07:11:27.250 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T07:11:27.273 INFO:teuthology.orchestra.run.smithi031.stdout:18.2.4-820.g48a7de7e.el9 2024-09-13T07:11:27.274 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820.g48a7de7e.el9 2024-09-13T07:11:27.274 INFO:teuthology.task.install:The correct ceph version 18.2.4-820.g48a7de7e is installed. 2024-09-13T07:11:27.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T07:11:27.431 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T07:11:27.457 INFO:teuthology.orchestra.run.smithi184.stdout:18.2.4-820.g48a7de7e.el9 2024-09-13T07:11:27.457 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820.g48a7de7e.el9 2024-09-13T07:11:27.457 INFO:teuthology.task.install:The correct ceph version 18.2.4-820.g48a7de7e is installed. 2024-09-13T07:11:27.459 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T07:11:27.459 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:27.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T07:11:27.487 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:11:27.487 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T07:11:27.529 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T07:11:27.529 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:27.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T07:11:27.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T07:11:27.622 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:11:27.622 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T07:11:27.651 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T07:11:27.719 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T07:11:27.719 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:27.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T07:11:27.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T07:11:27.813 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:11:27.813 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T07:11:27.841 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T07:11:27.910 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T07:11:27.910 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:27.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T07:11:27.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T07:11:28.003 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:11:28.003 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T07:11:28.030 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T07:11:28.097 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T07:11:28.169 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T07:11:28.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-09-13T07:11:28.171 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2024-09-13T07:11:28.198 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T07:11:28.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T07:11:28.238 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T07:11:28.264 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T07:11:28.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T07:11:28.303 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T07:11:28.339 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T07:11:28.339 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T07:11:28.339 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900909', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/EVN3lNe4Dh4k/3ZuykzC+Pnw87DwnhID5eEtcodpIhz5vtk789fTzLMtByYtGd+V7kkP2FqRyY2QJ5pIt/iU=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPQOOEitnzJ/qSjJcwV/cK+1PcUH6n8Humuo1Spa2wxi3YcXOrGsv77v244sD0K965lw7waQj4ULutAjKSvWMVc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T07:11:28.339 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T07:11:28.369 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T07:11:28.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T07:11:28.425 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T07:11:28.450 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:28.451 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T07:11:28.505 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T07:11:28.505 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 07:11:23.725409007 +0000 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 07:09:03.004684950 +0000 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 07:09:03.004684950 +0000 2024-09-13T07:11:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-13 07:09:03.004684950 +0000 2024-09-13T07:11:28.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T07:11:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T07:11:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T07:11:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000205263 s, 2.5 MB/s 2024-09-13T07:11:28.627 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T07:11:28.684 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 07:11:23.725409007 +0000 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 07:09:03.339679614 +0000 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 07:09:03.339679614 +0000 2024-09-13T07:11:28.740 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-13 07:09:03.339679614 +0000 2024-09-13T07:11:28.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T07:11:28.803 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T07:11:28.803 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T07:11:28.804 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000188972 s, 2.7 MB/s 2024-09-13T07:11:28.805 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T07:11:28.861 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-13T07:11:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 07:11:23.726408991 +0000 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 07:09:03.680674184 +0000 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 07:09:03.680674184 +0000 2024-09-13T07:11:28.917 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-13 07:09:03.680674184 +0000 2024-09-13T07:11:28.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T07:11:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T07:11:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T07:11:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000221333 s, 2.3 MB/s 2024-09-13T07:11:28.982 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T07:11:29.039 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 07:11:23.726408991 +0000 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 07:09:04.004669023 +0000 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 07:09:04.004669023 +0000 2024-09-13T07:11:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-13 07:09:04.004669023 +0000 2024-09-13T07:11:29.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T07:11:29.158 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T07:11:29.158 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T07:11:29.159 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000234851 s, 2.2 MB/s 2024-09-13T07:11:29.160 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T07:11:29.216 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:29.217 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:11:29.217 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T07:11:29.236 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T07:11:29.236 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-09-13T07:11:29.294 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T07:11:29.294 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:29.294 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T07:11:29.294 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:29.294 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:29.294 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-13 07:11:22.022000238 +0000 2024-09-13T07:11:29.294 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-13 07:09:01.653454603 +0000 2024-09-13T07:11:29.295 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-13 07:09:01.653454603 +0000 2024-09-13T07:11:29.295 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-13 07:09:01.653454603 +0000 2024-09-13T07:11:29.295 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T07:11:29.363 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-13T07:11:29.363 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-13T07:11:29.363 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000208858 s, 2.5 MB/s 2024-09-13T07:11:29.364 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T07:11:29.425 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-13 07:11:22.022000238 +0000 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-13 07:09:01.981447919 +0000 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-13 07:09:01.981447919 +0000 2024-09-13T07:11:29.484 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-13 07:09:01.981447919 +0000 2024-09-13T07:11:29.485 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T07:11:29.552 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-13T07:11:29.552 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-13T07:11:29.552 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000237983 s, 2.2 MB/s 2024-09-13T07:11:29.554 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T07:11:29.613 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-09-13T07:11:29.672 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T07:11:29.672 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:29.672 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T07:11:29.672 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:29.673 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:29.673 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-13 07:11:22.022000238 +0000 2024-09-13T07:11:29.673 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-13 07:09:02.300441418 +0000 2024-09-13T07:11:29.673 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-13 07:09:02.300441418 +0000 2024-09-13T07:11:29.673 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-13 07:09:02.300441418 +0000 2024-09-13T07:11:29.673 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T07:11:29.740 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-13T07:11:29.740 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-13T07:11:29.740 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000259248 s, 2.0 MB/s 2024-09-13T07:11:29.742 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T07:11:29.801 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-09-13 07:11:22.023000218 +0000 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-09-13 07:09:02.622434856 +0000 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-09-13 07:09:02.622434856 +0000 2024-09-13T07:11:29.861 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-09-13 07:09:02.622434856 +0000 2024-09-13T07:11:29.862 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T07:11:29.928 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-09-13T07:11:29.928 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-09-13T07:11:29.928 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000207359 s, 2.5 MB/s 2024-09-13T07:11:29.929 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T07:11:29.989 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:29.989 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T07:11:29.989 INFO:tasks.ceph:Generating config... 2024-09-13T07:11:29.991 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-09-13T07:11:29.991 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-09-13T07:11:29.991 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-09-13T07:11:29.991 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T07:11:29.991 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T07:11:29.991 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T07:11:29.991 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T07:11:29.991 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T07:11:29.991 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T07:11:29.991 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T07:11:29.991 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T07:11:29.991 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T07:11:29.991 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-13T07:11:29.991 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T07:11:29.992 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T07:11:29.993 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T07:11:29.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T07:11:30.053 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T07:11:30.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T07:11:30.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T07:11:30.130 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]'), ('mon.b', '172.21.15.184')] 2024-09-13T07:11:30.131 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.184', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T07:11:30.132 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:30.132 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T07:11:30.187 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --add b 172.21.15.184 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid f667c631-bb9d-47bf-97c6-7217d8893244 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:fsid f667c631-bb9d-47bf-97c6-7217d8893244 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-09-13T07:11:30.276723+0000 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-13T07:11:30.276723+0000 2024-09-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 16 (pacific) 2024-09-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-09-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2024-09-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2024-09-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T07:11:30.277 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T07:11:30.293 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f667c631-bb9d-47bf-97c6-7217d8893244... 2024-09-13T07:11:30.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T07:11:30.336 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-09-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-09-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-09-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.184" 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: mon client directed command retry = 5 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on pool no app = False 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: ms inject socket failures = 5000 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = f667c631-bb9d-47bf-97c6-7217d8893244 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression algorithm = lz4 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression mode = aggressive 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2024-09-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: osd max markdown count = 1000 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: osd shutdown pgref assert = True 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: rbd qos iops burst = 100 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: rbd qos iops limit = 50 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: rbd qos schedule tick min = 100 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-09-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2024-09-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2024-09-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.184" 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: mon client directed command retry = 5 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on pool no app = False 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: ms inject socket failures = 5000 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = f667c631-bb9d-47bf-97c6-7217d8893244 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2024-09-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore compression algorithm = lz4 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore compression mode = aggressive 2024-09-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 1/20 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 1/20 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 4/10 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: osd blocked scrub grace period = 3600 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: osd max markdown count = 1000 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: osd shutdown pgref assert = True 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2024-09-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2024-09-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: rbd qos iops burst = 100 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: rbd qos iops limit = 50 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout: rbd qos schedule tick min = 100 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2024-09-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2024-09-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2024-09-13T07:11:30.399 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T07:11:30.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T07:11:30.494 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T07:11:30.494 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:30.494 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T07:11:30.510 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:30.510 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T07:11:30.566 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:11:30.566 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:30.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T07:11:30.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T07:11:30.641 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:11:30.642 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:30.698 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:11:30.698 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:11:30.698 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T07:11:30.698 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T07:11:30.740 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:11:30.740 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:30.800 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T07:11:30.800 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T07:11:30.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T07:11:30.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T07:11:30.926 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T07:11:30.926 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T07:11:30.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T07:11:30.982 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T07:11:31.047 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T07:11:31.047 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T07:11:31.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T07:11:31.072 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:31.072 INFO:tasks.ceph:role: osd.0 2024-09-13T07:11:31.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:11:31.072 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:31.645 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T07:11:31.647 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T07:11:31.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T07:11:31.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T07:11:31.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T07:11:31.871 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:31.871 INFO:tasks.ceph:role: osd.1 2024-09-13T07:11:31.871 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:11:31.871 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T07:11:31.945 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:31.946 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:31.946 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:31.946 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:31.946 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:32.410 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T07:11:32.412 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T07:11:32.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T07:11:32.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T07:11:32.535 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T07:11:32.604 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:32.604 INFO:tasks.ceph:role: osd.2 2024-09-13T07:11:32.604 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:11:32.604 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T07:11:32.679 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:32.680 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:33.171 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T07:11:33.174 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T07:11:33.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T07:11:33.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T07:11:33.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T07:11:33.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:33.390 INFO:tasks.ceph:role: osd.3 2024-09-13T07:11:33.390 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:11:33.390 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:33.465 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:33.466 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:34.000 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T07:11:34.002 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T07:11:34.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T07:11:34.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T07:11:34.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:34.253+0000 7f158b6ef640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T07:11:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:34.253+0000 7f158b6ef640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T07:11:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:34.253+0000 7f158b6ef640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T07:11:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:34.254+0000 7f158b6ef640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T07:11:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:34.254+0000 7f158b6ef640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T07:11:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:34.254+0000 7f158b6ef640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T07:11:37.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T07:11:37.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:37.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:37.253+0000 7f4da06b0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T07:11:37.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:37.254+0000 7f4da06b0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T07:11:37.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:37.254+0000 7f4da06b0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T07:11:37.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:37.254+0000 7f4da06b0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T07:11:37.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:37.254+0000 7f4da06b0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T07:11:37.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:37.254+0000 7f4da06b0640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T07:11:40.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T07:11:40.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:40.236+0000 7f37a2a19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T07:11:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:40.236+0000 7f37a2a19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T07:11:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:40.236+0000 7f37a2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T07:11:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:40.236+0000 7f37a2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T07:11:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:40.236+0000 7f37a2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T07:11:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:40.237+0000 7f37a2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T07:11:43.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T07:11:43.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:43.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:43.211+0000 7fe699c16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T07:11:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:43.211+0000 7fe699c16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T07:11:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:43.211+0000 7fe699c16640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T07:11:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:43.211+0000 7fe699c16640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T07:11:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:43.211+0000 7fe699c16640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T07:11:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T07:11:43.212+0000 7fe699c16640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T07:11:46.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T07:11:46.099 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T07:11:46.130 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:46.130 INFO:tasks.ceph:role: osd.4 2024-09-13T07:11:46.130 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:11:46.130 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T07:11:46.205 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:46.205 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:46.205 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:46.205 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:46.205 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:46.205 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:46.206 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:46.206 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:46.206 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:46.206 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:46.668 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-13T07:11:46.670 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-13T07:11:46.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T07:11:46.789 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T07:11:46.825 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T07:11:46.893 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:46.893 INFO:tasks.ceph:role: osd.5 2024-09-13T07:11:46.893 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:11:46.894 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:46.969 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:47.438 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-13T07:11:47.440 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-13T07:11:47.440 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T07:11:47.512 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T07:11:47.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T07:11:47.645 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:47.645 INFO:tasks.ceph:role: osd.6 2024-09-13T07:11:47.645 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:11:47.646 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T07:11:47.716 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:47.717 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:48.200 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-13T07:11:48.202 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-13T07:11:48.203 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T07:11:48.297 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T07:11:48.327 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T07:11:48.391 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:11:48.392 INFO:tasks.ceph:role: osd.7 2024-09-13T07:11:48.392 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:11:48.392 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T07:11:48.464 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:11:48.464 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:11:48.464 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:11:48.464 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T07:11:48.465 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:11:48.465 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:11:48.465 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:11:48.465 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T07:11:48.465 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:11:48.465 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:11:48.989 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-09-13T07:11:48.991 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-09-13T07:11:48.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T07:11:49.133 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T07:11:49.165 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:49.274 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:49.274+0000 7f14ba411640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T07:11:49.276 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:49.277+0000 7f14ba411640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T07:11:49.276 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:49.277+0000 7f14ba411640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T07:11:49.276 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:49.277+0000 7f14ba411640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T07:11:49.276 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:49.277+0000 7f14ba411640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T07:11:49.283 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:49.284+0000 7f14ba411640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T07:11:52.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T07:11:52.169 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:52.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:52.258+0000 7fc7990d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T07:11:52.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:52.258+0000 7fc7990d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T07:11:52.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:52.258+0000 7fc7990d6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T07:11:52.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:52.258+0000 7fc7990d6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T07:11:52.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:52.258+0000 7fc7990d6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T07:11:52.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:52.259+0000 7fc7990d6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T07:11:55.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T07:11:55.127 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:55.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:55.214+0000 7fcbdac17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T07:11:55.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:55.214+0000 7fcbdac17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T07:11:55.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:55.214+0000 7fcbdac17640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T07:11:55.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:55.214+0000 7fcbdac17640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T07:11:55.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:55.214+0000 7fcbdac17640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T07:11:55.214 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:55.215+0000 7fcbdac17640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T07:11:58.067 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T07:11:58.093 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:11:58.179 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:58.179+0000 7f4df6e0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T07:11:58.179 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:58.180+0000 7f4df6e0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T07:11:58.179 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:58.180+0000 7f4df6e0b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T07:11:58.179 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:58.180+0000 7f4df6e0b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T07:11:58.180 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:58.180+0000 7f4df6e0b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T07:11:58.180 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:11:58.180+0000 7f4df6e0b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T07:12:01.033 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T07:12:01.062 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T07:12:01.063 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:01.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T07:12:01.094 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:01.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T07:12:01.164 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:01.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T07:12:01.234 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:01.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T07:12:01.302 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:01.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T07:12:01.370 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:01.370 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T07:12:01.401 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:01.401 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T07:12:01.471 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:01.471 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T07:12:01.541 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:01.541 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T07:12:01.611 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:01.611 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T07:12:01.681 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:01.681 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T07:12:01.700 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:01.700 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T07:12:01.740 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T07:12:01.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T07:12:01.743 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T07:12:01.783 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2024-09-13T07:12:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCi5eNmdiPmMhAACSAvI6+/IpbrYos6anMK6A== 2024-09-13T07:12:01.783 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-09-13T07:12:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCm5eNmgAwsDxAAyxwA6WS4jlsufHrkw8h0DA== 2024-09-13T07:12:01.783 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCp5eNmS+ouDxAA8ylLQ3lCU5BBylgHvd+bCQ== 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCs5eNmb+4mDhAAhGENdmC2CMJT6t2H+965SQ== 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCv5eNmF0ioDBAA+6XGNrFOpMSELlqRzLZ2Rw== 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCi5eNmk8AgNxAAE+kD9MT/QcVBDXhfr8RPAQ== 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQC15eNmcH1vEBAAR9vC13ZvqGJxuUfHuYPn0Q== 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQC45eNmxnFxDxAALVFOX238cP67pb9rUo6IuA== 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2024-09-13T07:12:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQC75eNmih7VDBAALuajsk/ZhUfmuNi7lvhjBQ== 2024-09-13T07:12:01.785 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2024-09-13T07:12:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQC+5eNmeAnIChAAMDZl+PfOxbb5W7MK5qTONw== 2024-09-13T07:12:01.785 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-13T07:12:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCi5eNmSJzdORAAXlgQ7NByU5h9yALyOQPIpQ== 2024-09-13T07:12:01.785 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2024-09-13T07:12:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCj5eNm5sr6ARAAT2sQaeb8rO5nXQNDn5+2ow== 2024-09-13T07:12:01.808 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2024-09-13T07:12:01.808 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCi5eNmdiPmMhAACSAvI6+/IpbrYos6anMK6A== 2024-09-13T07:12:01.808 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCm5eNmgAwsDxAAyxwA6WS4jlsufHrkw8h0DA== 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCp5eNmS+ouDxAA8ylLQ3lCU5BBylgHvd+bCQ== 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCs5eNmb+4mDhAAhGENdmC2CMJT6t2H+965SQ== 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCv5eNmF0ioDBAA+6XGNrFOpMSELlqRzLZ2Rw== 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCi5eNmk8AgNxAAE+kD9MT/QcVBDXhfr8RPAQ== 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQC15eNmcH1vEBAAR9vC13ZvqGJxuUfHuYPn0Q== 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQC45eNmxnFxDxAALVFOX238cP67pb9rUo6IuA== 2024-09-13T07:12:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2024-09-13T07:12:01.810 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQC75eNmih7VDBAALuajsk/ZhUfmuNi7lvhjBQ== 2024-09-13T07:12:01.810 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2024-09-13T07:12:01.810 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQC+5eNmeAnIChAAMDZl+PfOxbb5W7MK5qTONw== 2024-09-13T07:12:01.810 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2024-09-13T07:12:01.810 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCi5eNmSJzdORAAXlgQ7NByU5h9yALyOQPIpQ== 2024-09-13T07:12:01.810 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2024-09-13T07:12:01.810 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCj5eNm5sr6ARAAT2sQaeb8rO5nXQNDn5+2ow== 2024-09-13T07:12:01.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:12:01.829 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:12:01.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:01.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:01.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:01.969 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.023 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:12:02.128 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:12:02.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.181 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.234 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.286 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.339 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:12:02.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:12:02.390 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:12:02.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:12:02.445 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:12:02.498 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T07:12:02.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T07:12:02.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T07:12:02.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T07:12:02.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T07:12:02.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T07:12:02.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T07:12:02.835 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T07:12:02.861 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T07:12:02.956 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T07:12:02.981 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:12:02.983 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:12:03.036 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T07:12:03.037 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T07:12:03.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T07:12:03.039 INFO:tasks.ceph.mon.a:Started 2024-09-13T07:12:03.040 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T07:12:03.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T07:12:03.042 INFO:tasks.ceph.mon.c:Started 2024-09-13T07:12:03.042 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T07:12:03.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T07:12:03.078 INFO:tasks.ceph.mon.b:Started 2024-09-13T07:12:03.078 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T07:12:03.079 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T07:12:03.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T07:12:03.081 INFO:tasks.ceph.mgr.y:Started 2024-09-13T07:12:03.081 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T07:12:03.081 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T07:12:03.083 INFO:tasks.ceph.mgr.x:Started 2024-09-13T07:12:03.083 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T07:12:03.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T07:12:08.440 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-13T07:12:08.452 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T07:12:08.452 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T07:12:08.453 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:08.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T07:12:08.472 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:08.472+0000 7f680ca49040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T07:12:08.480 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:08.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T07:12:08.513 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:08.513+0000 7f680ca49040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T07:12:08.549 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:08.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T07:12:08.615 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T07:12:08.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T07:12:08.662 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:08.663+0000 7f680ca49040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T07:12:08.681 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:08.681 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T07:12:08.706 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:08.706 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T07:12:08.714 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:08.714+0000 7f680ca49040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T07:12:08.738 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:08.739 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T07:12:08.804 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-09-13T07:12:08.804 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T07:12:08.869 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new c29fb342-d45b-48c1-9694-dd01e2a01325 0 2024-09-13T07:12:08.887 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:08.888+0000 7f680ca49040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:12:09.002 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:09.003+0000 7f680ca49040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T07:12:09.190 INFO:teuthology.orchestra.run.smithi184.stderr:2024-09-13T07:12:09.191+0000 7f2b077fe640 0 --2- 172.21.15.184:0/2383676444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b0806f980 0x7f2b081343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T07:12:09.263 INFO:teuthology.orchestra.run.smithi184.stdout:0 2024-09-13T07:12:09.274 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 4b979b5c-a78b-461d-b795-f6f5c25cf652 1 2024-09-13T07:12:09.277 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:09.278+0000 7f13b922d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T07:12:09.321 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:09.322+0000 7f13b922d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T07:12:09.330 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:09.331+0000 7f680ca49040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T07:12:09.387 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:09.387+0000 7f680ca49040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T07:12:09.470 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:09.471+0000 7f13b922d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T07:12:09.470 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-09-13T07:12:09.481 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 11259f54-1134-495f-9281-d5132aff2780 2 2024-09-13T07:12:09.522 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:09.523+0000 7f13b922d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T07:12:09.551 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:09.551+0000 7f680ca49040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T07:12:09.679 INFO:teuthology.orchestra.run.smithi184.stdout:2 2024-09-13T07:12:09.688 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:09.689+0000 7f13b922d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:12:09.690 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new a11be2be-3803-4a23-bbf8-40ea110ccf41 3 2024-09-13T07:12:09.803 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:09.803+0000 7f13b922d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T07:12:09.868 INFO:teuthology.orchestra.run.smithi184.stdout:3 2024-09-13T07:12:09.879 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 2b6941d9-29f5-4836-a61c-47ff27a7172e 4 2024-09-13T07:12:10.060 INFO:teuthology.orchestra.run.smithi184.stdout:4 2024-09-13T07:12:10.071 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 17e3a02d-28e3-4126-a8f2-c7783e524d62 5 2024-09-13T07:12:10.139 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:10.140+0000 7f13b922d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T07:12:10.191 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:10.192+0000 7f13b922d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T07:12:10.249 INFO:teuthology.orchestra.run.smithi184.stdout:5 2024-09-13T07:12:10.260 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new ddf29f57-7733-42fa-94ed-015a529fb689 6 2024-09-13T07:12:10.347 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:10.348+0000 7f13b922d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T07:12:10.441 INFO:teuthology.orchestra.run.smithi184.stdout:6 2024-09-13T07:12:10.452 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new d91e2eed-3bfb-4eb0-ac30-18770c106742 7 2024-09-13T07:12:10.561 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:10.561+0000 7f680ca49040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T07:12:10.612 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:10.613+0000 7f680ca49040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T07:12:10.629 INFO:teuthology.orchestra.run.smithi184.stdout:7 2024-09-13T07:12:10.640 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T07:12:10.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T07:12:10.642 INFO:tasks.ceph.osd.0:Started 2024-09-13T07:12:10.642 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T07:12:10.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T07:12:10.644 INFO:tasks.ceph.osd.1:Started 2024-09-13T07:12:10.645 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T07:12:10.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T07:12:10.647 INFO:tasks.ceph.osd.2:Started 2024-09-13T07:12:10.647 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T07:12:10.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T07:12:10.649 INFO:tasks.ceph.osd.3:Started 2024-09-13T07:12:10.649 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T07:12:10.650 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T07:12:10.652 INFO:tasks.ceph.osd.4:Started 2024-09-13T07:12:10.652 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T07:12:10.652 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T07:12:10.654 INFO:tasks.ceph.osd.5:Started 2024-09-13T07:12:10.654 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T07:12:10.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T07:12:10.656 INFO:tasks.ceph.osd.6:Started 2024-09-13T07:12:10.656 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T07:12:10.656 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T07:12:10.658 INFO:tasks.ceph.osd.7:Started 2024-09-13T07:12:10.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:12:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:10.862 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:12:10.626165+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:12:10.872 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T07:12:10.872 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T07:12:11.021 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.022+0000 7f680ca49040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T07:12:11.105 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.105+0000 7f680ca49040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T07:12:11.207 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.207+0000 7f680ca49040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:12:11.247 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:11.247+0000 7f35dd334640 -1 Falling back to public interface 2024-09-13T07:12:11.252 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.253+0000 7f680ca49040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T07:12:11.256 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:11.257+0000 7f07060da640 -1 Falling back to public interface 2024-09-13T07:12:11.262 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:11.262+0000 7f684ea0d640 -1 Falling back to public interface 2024-09-13T07:12:11.262 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:11.262+0000 7f0acaa0d640 -1 Falling back to public interface 2024-09-13T07:12:11.262 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:11.262+0000 7f884e018640 -1 Falling back to public interface 2024-09-13T07:12:11.262 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:11.262+0000 7f684d338640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-09-13T07:12:11.263 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:11.264+0000 7f884c993640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6819/0: (98) Address already in use 2024-09-13T07:12:11.271 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:11.272+0000 7f0901edc640 -1 Falling back to public interface 2024-09-13T07:12:11.278 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:11.279+0000 7fd396628640 -1 Falling back to public interface 2024-09-13T07:12:11.293 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:11.294+0000 7fa9ec1dd640 -1 Falling back to public interface 2024-09-13T07:12:11.367 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.368+0000 7f680ca49040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T07:12:11.368 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:11.368+0000 7f13b922d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T07:12:11.419 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.419+0000 7f680ca49040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T07:12:11.421 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:11.422+0000 7f13b922d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T07:12:11.462 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.463+0000 7f680ca49040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T07:12:11.573 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.574+0000 7f680ca49040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T07:12:11.624 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.624+0000 7f680ca49040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T07:12:11.672 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.673+0000 7f680ca49040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T07:12:11.677 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T07:12:11.738 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.739+0000 7f680ca49040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T07:12:11.781 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.782+0000 7f680ca49040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T07:12:11.810 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:11.811+0000 7f13b922d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T07:12:11.826 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:11.827+0000 7f680ca49040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T07:12:11.837 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-13T07:12:11.838 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:12:10.626165+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:12:11.847 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T07:12:11.899 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:11.900+0000 7f13b922d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T07:12:12.009 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.010+0000 7f13b922d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:12:12.058 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.058+0000 7f13b922d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T07:12:12.174 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.175+0000 7f13b922d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T07:12:12.230 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.230+0000 7f13b922d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T07:12:12.276 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.277+0000 7f13b922d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T07:12:12.393 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.394+0000 7f13b922d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T07:12:12.445 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.446+0000 7f13b922d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T07:12:12.458 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:12.458+0000 7f680ca49040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T07:12:12.505 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.505+0000 7f13b922d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T07:12:12.575 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.576+0000 7f13b922d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T07:12:12.622 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.622+0000 7f13b922d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T07:12:12.667 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:12.668+0000 7f13b922d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T07:12:13.338 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T07:12:13.338+0000 7f13b922d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T07:12:13.472 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:13.471+0000 7f67adce8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T07:12:13.474 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-09-13T07:12:13.474+0000 7f67adce8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T07:12:14.713 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:14.713+0000 7f35dd334640 -1 osd.4 0 log_to_monitors true 2024-09-13T07:12:14.717 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:14.718+0000 7f07060da640 -1 osd.2 0 log_to_monitors true 2024-09-13T07:12:14.756 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:14.756+0000 7f684ea0d640 -1 osd.7 0 log_to_monitors true 2024-09-13T07:12:14.756 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:14.756+0000 7f884e018640 -1 osd.5 0 log_to_monitors true 2024-09-13T07:12:14.781 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:14.782+0000 7f0901edc640 -1 osd.3 0 log_to_monitors true 2024-09-13T07:12:14.786 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:14.787+0000 7f0acaa0d640 -1 osd.0 0 log_to_monitors true 2024-09-13T07:12:14.786 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:14.787+0000 7fd396628640 -1 osd.1 0 log_to_monitors true 2024-09-13T07:12:14.801 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:14.801+0000 7fa9ec1dd640 -1 osd.6 0 log_to_monitors true 2024-09-13T07:12:17.563 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:17.562+0000 7f35d92c1640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T07:12:17.575 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:17.575+0000 7f35d10e2640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:17.599 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:17.599+0000 7f8849977640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T07:12:17.609 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:17.609+0000 7f8841798640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:17.616 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:17.616+0000 7fa9e8981640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T07:12:17.616 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:17.616+0000 7f684a31c640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T07:12:17.631 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:17.631+0000 7fa9dff8b640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:17.631 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:17.631+0000 7f684213d640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:17.694 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:17.694+0000 7f0702067640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T07:12:17.704 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:17.705+0000 7f0ac631c640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T07:12:17.707 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:17.708+0000 7f06f9e88640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:17.711 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:17.712+0000 7f08fde69640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T07:12:17.711 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:17.712+0000 7fd3925b5640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T07:12:17.719 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:17.720+0000 7f08f5c8a640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:17.720 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:17.720+0000 7f0abe13d640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:17.723 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:17.724+0000 7fd38a3d6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:12:18.654 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T07:12:18.942 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-13T07:12:18.942 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":13,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:12:18.566379+0000","last_up_change":"2024-09-13T07:12:18.566379+0000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6809","nonce":1709347595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6811","nonce":1709347595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6815","nonce":1709347595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6813","nonce":1709347595}]},"public_addr":"172.21.15.31:6809/1709347595","cluster_addr":"172.21.15.31:6811/1709347595","heartbeat_back_addr":"172.21.15.31:6815/1709347595","heartbeat_front_addr":"172.21.15.31:6813/1709347595","state":["exists","up"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6825","nonce":3300827638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6827","nonce":3300827638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6831","nonce":3300827638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6829","nonce":3300827638}]},"public_addr":"172.21.15.31:6825/3300827638","cluster_addr":"172.21.15.31:6827/3300827638","heartbeat_back_addr":"172.21.15.31:6831/3300827638","heartbeat_front_addr":"172.21.15.31:6829/3300827638","state":["exists","up"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6801","nonce":3977233096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6803","nonce":3977233096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6807","nonce":3977233096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6805","nonce":3977233096}]},"public_addr":"172.21.15.31:6801/3977233096","cluster_addr":"172.21.15.31:6803/3977233096","heartbeat_back_addr":"172.21.15.31:6807/3977233096","heartbeat_front_addr":"172.21.15.31:6805/3977233096","state":["exists","up"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6817","nonce":3290545828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6819","nonce":3290545828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6823","nonce":3290545828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6821","nonce":3290545828}]},"public_addr":"172.21.15.31:6817/3290545828","cluster_addr":"172.21.15.31:6819/3290545828","heartbeat_back_addr":"172.21.15.31:6823/3290545828","heartbeat_front_addr":"172.21.15.31:6821/3290545828","state":["exists","up"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6801","nonce":3543694628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6803","nonce":3543694628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6807","nonce":3543694628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6805","nonce":3543694628}]},"public_addr":"172.21.15.184:6801/3543694628","cluster_addr":"172.21.15.184:6803/3543694628","heartbeat_back_addr":"172.21.15.184:6807/3543694628","heartbeat_front_addr":"172.21.15.184:6805/3543694628","state":["exists","up"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6810","nonce":951246042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6814","nonce":951246042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6821","nonce":951246042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6818","nonce":951246042}]},"public_addr":"172.21.15.184:6810/951246042","cluster_addr":"172.21.15.184:6814/951246042","heartbeat_back_addr":"172.21.15.184:6821/951246042","heartbeat_front_addr":"172.21.15.184:6818/951246042","state":["exists","up"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6825","nonce":1941674146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6827","nonce":1941674146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6831","nonce":1941674146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6829","nonce":1941674146}]},"public_addr":"172.21.15.184:6825/1941674146","cluster_addr":"172.21.15.184:6827/1941674146","heartbeat_back_addr":"172.21.15.184:6831/1941674146","heartbeat_front_addr":"172.21.15.184:6829/1941674146","state":["exists","up"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6811","nonce":322162340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6815","nonce":322162340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6823","nonce":322162340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6819","nonce":322162340}]},"public_addr":"172.21.15.184:6811/322162340","cluster_addr":"172.21.15.184:6815/322162340","heartbeat_back_addr":"172.21.15.184:6823/322162340","heartbeat_front_addr":"172.21.15.184:6819/322162340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:12:18.953 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T07:12:18.954 INFO:tasks.ceph:Creating RBD pool 2024-09-13T07:12:18.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T07:12:19.575 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2024-09-13T07:12:19.588 DEBUG:teuthology.orchestra.run.smithi031:> rbd --cluster ceph pool init rbd 2024-09-13T07:12:22.599 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T07:12:22.600 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T07:12:22.601 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T07:12:22.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T07:12:22.602 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T07:12:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:22.950 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":785568159},{"type":"v1","addr":"172.21.15.184:6833","nonce":785568159}]},"active_addr":"172.21.15.184:6833/785568159","active_change":"2024-09-13T07:12:12.462408+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4125,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":2494166651}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":366248041}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":1317907933}]}]} 2024-09-13T07:12:22.956 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T07:12:22.956 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T07:12:22.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:12:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:23.245 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:12:22.586643+0000","last_up_change":"2024-09-13T07:12:18.566379+0000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T07:12:19.268005+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T07:12:20.513150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6809","nonce":1709347595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6811","nonce":1709347595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6815","nonce":1709347595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6813","nonce":1709347595}]},"public_addr":"172.21.15.31:6809/1709347595","cluster_addr":"172.21.15.31:6811/1709347595","heartbeat_back_addr":"172.21.15.31:6815/1709347595","heartbeat_front_addr":"172.21.15.31:6813/1709347595","state":["exists","up"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6825","nonce":3300827638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6827","nonce":3300827638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6831","nonce":3300827638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6829","nonce":3300827638}]},"public_addr":"172.21.15.31:6825/3300827638","cluster_addr":"172.21.15.31:6827/3300827638","heartbeat_back_addr":"172.21.15.31:6831/3300827638","heartbeat_front_addr":"172.21.15.31:6829/3300827638","state":["exists","up"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6801","nonce":3977233096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6803","nonce":3977233096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6807","nonce":3977233096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6805","nonce":3977233096}]},"public_addr":"172.21.15.31:6801/3977233096","cluster_addr":"172.21.15.31:6803/3977233096","heartbeat_back_addr":"172.21.15.31:6807/3977233096","heartbeat_front_addr":"172.21.15.31:6805/3977233096","state":["exists","up"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6817","nonce":3290545828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6819","nonce":3290545828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6823","nonce":3290545828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6821","nonce":3290545828}]},"public_addr":"172.21.15.31:6817/3290545828","cluster_addr":"172.21.15.31:6819/3290545828","heartbeat_back_addr":"172.21.15.31:6823/3290545828","heartbeat_front_addr":"172.21.15.31:6821/3290545828","state":["exists","up"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6801","nonce":3543694628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6803","nonce":3543694628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6807","nonce":3543694628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6805","nonce":3543694628}]},"public_addr":"172.21.15.184:6801/3543694628","cluster_addr":"172.21.15.184:6803/3543694628","heartbeat_back_addr":"172.21.15.184:6807/3543694628","heartbeat_front_addr":"172.21.15.184:6805/3543694628","state":["exists","up"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6810","nonce":951246042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6814","nonce":951246042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6821","nonce":951246042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6818","nonce":951246042}]},"public_addr":"172.21.15.184:6810/951246042","cluster_addr":"172.21.15.184:6814/951246042","heartbeat_back_addr":"172.21.15.184:6821/951246042","heartbeat_front_addr":"172.21.15.184:6818/951246042","state":["exists","up"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6825","nonce":1941674146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6827","nonce":1941674146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6831","nonce":1941674146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6829","nonce":1941674146}]},"public_addr":"172.21.15.184:6825/1941674146","cluster_addr":"172.21.15.184:6827/1941674146","heartbeat_back_addr":"172.21.15.184:6831/1941674146","heartbeat_front_addr":"172.21.15.184:6829/1941674146","state":["exists","up"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6811","nonce":322162340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6815","nonce":322162340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6823","nonce":322162340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6819","nonce":322162340}]},"public_addr":"172.21.15.184:6811/322162340","cluster_addr":"172.21.15.184:6815/322162340","heartbeat_back_addr":"172.21.15.184:6823/322162340","heartbeat_front_addr":"172.21.15.184:6819/322162340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.805494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.817444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.712083+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.739958+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.720767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.715944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.788853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.809325+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:12:23.257 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T07:12:23.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:12:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:23.544 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:12:22.586643+0000","last_up_change":"2024-09-13T07:12:18.566379+0000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T07:12:19.268005+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T07:12:20.513150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6809","nonce":1709347595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6811","nonce":1709347595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6815","nonce":1709347595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6813","nonce":1709347595}]},"public_addr":"172.21.15.31:6809/1709347595","cluster_addr":"172.21.15.31:6811/1709347595","heartbeat_back_addr":"172.21.15.31:6815/1709347595","heartbeat_front_addr":"172.21.15.31:6813/1709347595","state":["exists","up"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6825","nonce":3300827638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6827","nonce":3300827638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6831","nonce":3300827638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6829","nonce":3300827638}]},"public_addr":"172.21.15.31:6825/3300827638","cluster_addr":"172.21.15.31:6827/3300827638","heartbeat_back_addr":"172.21.15.31:6831/3300827638","heartbeat_front_addr":"172.21.15.31:6829/3300827638","state":["exists","up"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6801","nonce":3977233096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6803","nonce":3977233096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6807","nonce":3977233096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6805","nonce":3977233096}]},"public_addr":"172.21.15.31:6801/3977233096","cluster_addr":"172.21.15.31:6803/3977233096","heartbeat_back_addr":"172.21.15.31:6807/3977233096","heartbeat_front_addr":"172.21.15.31:6805/3977233096","state":["exists","up"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6817","nonce":3290545828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6819","nonce":3290545828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6823","nonce":3290545828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6821","nonce":3290545828}]},"public_addr":"172.21.15.31:6817/3290545828","cluster_addr":"172.21.15.31:6819/3290545828","heartbeat_back_addr":"172.21.15.31:6823/3290545828","heartbeat_front_addr":"172.21.15.31:6821/3290545828","state":["exists","up"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6801","nonce":3543694628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6803","nonce":3543694628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6807","nonce":3543694628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6805","nonce":3543694628}]},"public_addr":"172.21.15.184:6801/3543694628","cluster_addr":"172.21.15.184:6803/3543694628","heartbeat_back_addr":"172.21.15.184:6807/3543694628","heartbeat_front_addr":"172.21.15.184:6805/3543694628","state":["exists","up"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6810","nonce":951246042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6814","nonce":951246042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6821","nonce":951246042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6818","nonce":951246042}]},"public_addr":"172.21.15.184:6810/951246042","cluster_addr":"172.21.15.184:6814/951246042","heartbeat_back_addr":"172.21.15.184:6821/951246042","heartbeat_front_addr":"172.21.15.184:6818/951246042","state":["exists","up"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6825","nonce":1941674146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6827","nonce":1941674146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6831","nonce":1941674146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6829","nonce":1941674146}]},"public_addr":"172.21.15.184:6825/1941674146","cluster_addr":"172.21.15.184:6827/1941674146","heartbeat_back_addr":"172.21.15.184:6831/1941674146","heartbeat_front_addr":"172.21.15.184:6829/1941674146","state":["exists","up"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6811","nonce":322162340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6815","nonce":322162340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6823","nonce":322162340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":322162340},{"type":"v1","addr":"172.21.15.184:6819","nonce":322162340}]},"public_addr":"172.21.15.184:6811/322162340","cluster_addr":"172.21.15.184:6815/322162340","heartbeat_back_addr":"172.21.15.184:6823/322162340","heartbeat_front_addr":"172.21.15.184:6819/322162340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.805494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.817444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.712083+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.739958+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.720767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.715944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.788853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.809325+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:12:23.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T07:12:23.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T07:12:23.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T07:12:23.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T07:12:23.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T07:12:23.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T07:12:23.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T07:12:23.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T07:12:23.804 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:23.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:12:23.808 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:23.808 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:23.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:12:23.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:12:23.810 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:23.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:12:23.812 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:23.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:12:23.813 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:23.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:12:23.814 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2024-09-13T07:12:23.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:12:23.815 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:23.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:12:24.381 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-09-13T07:12:24.382 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-09-13T07:12:24.396 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-09-13T07:12:24.396 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-09-13T07:12:24.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-09-13T07:12:24.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2024-09-13T07:12:24.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2024-09-13T07:12:24.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2024-09-13T07:12:24.428 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-09-13T07:12:24.428 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-09-13T07:12:24.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-09-13T07:12:24.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-09-13T07:12:24.453 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-09-13T07:12:24.455 INFO:teuthology.orchestra.run.smithi031.stdout:55834574852 2024-09-13T07:12:24.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-09-13T07:12:24.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-09-13T07:12:25.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:12:25.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:12:25.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:12:25.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:12:25.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:12:25.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:12:25.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:12:25.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:12:25.970 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:25.988 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:25.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-09-13T07:12:25.990 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-09-13T07:12:26.007 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.019 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:26.035 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:26.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2024-09-13T07:12:26.037 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.041 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:26.049 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:26.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2024-09-13T07:12:26.055 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.057 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-09-13T07:12:26.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-09-13T07:12:26.060 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2024-09-13T07:12:26.063 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-09-13T07:12:26.071 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.074 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2024-09-13T07:12:26.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-09-13T07:12:26.087 DEBUG:teuthology.parallel:result is None 2024-09-13T07:12:26.087 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T07:12:26.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:12:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:26.409 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:12:26.420 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-13T07:12:24.469258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217676,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757044,"statfs":{"total":773094113280,"available":772871213056,"internally_reserved":0,"allocated":1466368,"data_stored":1607262,"data_compressed":16928,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590999+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:12:22.590999+0000","last_peered":"2024-09-13T07:12:22.590999+0000","last_clean":"2024-09-13T07:12:22.590999+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:12:22.590999+0000","last_undegraded":"2024-09-13T07:12:22.590999+0000","last_fullsized":"2024-09-13T07:12:22.590999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590994+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:12:22.590994+0000","last_peered":"2024-09-13T07:12:22.590994+0000","last_clean":"2024-09-13T07:12:22.590994+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:12:22.590994+0000","last_undegraded":"2024-09-13T07:12:22.590994+0000","last_fullsized":"2024-09-13T07:12:22.590994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.863101+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:12:22.863101+0000","last_peered":"2024-09-13T07:12:22.863101+0000","last_clean":"2024-09-13T07:12:22.863101+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:12:22.863101+0000","last_undegraded":"2024-09-13T07:12:22.863101+0000","last_fullsized":"2024-09-13T07:12:22.863101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592265+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:12:22.592265+0000","last_peered":"2024-09-13T07:12:22.592265+0000","last_clean":"2024-09-13T07:12:22.592265+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:12:22.592265+0000","last_undegraded":"2024-09-13T07:12:22.592265+0000","last_fullsized":"2024-09-13T07:12:22.592265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:23.040450+0000","last_change":"2024-09-13T07:12:21.825606+0000","last_active":"2024-09-13T07:12:23.040450+0000","last_peered":"2024-09-13T07:12:23.040450+0000","last_clean":"2024-09-13T07:12:23.040450+0000","last_became_active":"2024-09-13T07:12:21.824606+0000","last_became_peered":"2024-09-13T07:12:21.824606+0000","last_unstale":"2024-09-13T07:12:23.040450+0000","last_undegraded":"2024-09-13T07:12:23.040450+0000","last_fullsized":"2024-09-13T07:12:23.040450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:05.013770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592051+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:12:22.592051+0000","last_peered":"2024-09-13T07:12:22.592051+0000","last_clean":"2024-09-13T07:12:22.592051+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:12:22.592051+0000","last_undegraded":"2024-09-13T07:12:22.592051+0000","last_fullsized":"2024-09-13T07:12:22.592051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.596135+0000","last_change":"2024-09-13T07:12:22.593317+0000","last_active":"2024-09-13T07:12:22.596135+0000","last_peered":"2024-09-13T07:12:22.596135+0000","last_clean":"2024-09-13T07:12:22.596135+0000","last_became_active":"2024-09-13T07:12:20.600877+0000","last_became_peered":"2024-09-13T07:12:20.600877+0000","last_unstale":"2024-09-13T07:12:22.596135+0000","last_undegraded":"2024-09-13T07:12:22.596135+0000","last_fullsized":"2024-09-13T07:12:22.596135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:57.330291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077577500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.742180+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:12:22.742180+0000","last_peered":"2024-09-13T07:12:22.742180+0000","last_clean":"2024-09-13T07:12:22.742180+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:12:22.742180+0000","last_undegraded":"2024-09-13T07:12:22.742180+0000","last_fullsized":"2024-09-13T07:12:22.742180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592787+0000","last_change":"2024-09-13T07:12:22.592787+0000","last_active":"2024-09-13T07:12:22.592787+0000","last_peered":"2024-09-13T07:12:22.592787+0000","last_clean":"2024-09-13T07:12:22.592787+0000","last_became_active":"2024-09-13T07:12:20.600886+0000","last_became_peered":"2024-09-13T07:12:20.600886+0000","last_unstale":"2024-09-13T07:12:22.592787+0000","last_undegraded":"2024-09-13T07:12:22.592787+0000","last_fullsized":"2024-09-13T07:12:22.592787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:18.571233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024777700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8464,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8464,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:12:26.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:12:26.705 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:26.706 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:12:26.718 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-13T07:12:26.469700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217632,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757088,"statfs":{"total":773094113280,"available":772871258112,"internally_reserved":0,"allocated":1466368,"data_stored":1607262,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000441"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590999+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:12:22.590999+0000","last_peered":"2024-09-13T07:12:22.590999+0000","last_clean":"2024-09-13T07:12:22.590999+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:12:22.590999+0000","last_undegraded":"2024-09-13T07:12:22.590999+0000","last_fullsized":"2024-09-13T07:12:22.590999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590994+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:12:22.590994+0000","last_peered":"2024-09-13T07:12:22.590994+0000","last_clean":"2024-09-13T07:12:22.590994+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:12:22.590994+0000","last_undegraded":"2024-09-13T07:12:22.590994+0000","last_fullsized":"2024-09-13T07:12:22.590994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.863101+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:12:22.863101+0000","last_peered":"2024-09-13T07:12:22.863101+0000","last_clean":"2024-09-13T07:12:22.863101+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:12:22.863101+0000","last_undegraded":"2024-09-13T07:12:22.863101+0000","last_fullsized":"2024-09-13T07:12:22.863101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592265+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:12:22.592265+0000","last_peered":"2024-09-13T07:12:22.592265+0000","last_clean":"2024-09-13T07:12:22.592265+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:12:22.592265+0000","last_undegraded":"2024-09-13T07:12:22.592265+0000","last_fullsized":"2024-09-13T07:12:22.592265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:23.040450+0000","last_change":"2024-09-13T07:12:21.825606+0000","last_active":"2024-09-13T07:12:23.040450+0000","last_peered":"2024-09-13T07:12:23.040450+0000","last_clean":"2024-09-13T07:12:23.040450+0000","last_became_active":"2024-09-13T07:12:21.824606+0000","last_became_peered":"2024-09-13T07:12:21.824606+0000","last_unstale":"2024-09-13T07:12:23.040450+0000","last_undegraded":"2024-09-13T07:12:23.040450+0000","last_fullsized":"2024-09-13T07:12:23.040450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:05.013770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592051+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:12:22.592051+0000","last_peered":"2024-09-13T07:12:22.592051+0000","last_clean":"2024-09-13T07:12:22.592051+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:12:22.592051+0000","last_undegraded":"2024-09-13T07:12:22.592051+0000","last_fullsized":"2024-09-13T07:12:22.592051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.596135+0000","last_change":"2024-09-13T07:12:22.593317+0000","last_active":"2024-09-13T07:12:22.596135+0000","last_peered":"2024-09-13T07:12:22.596135+0000","last_clean":"2024-09-13T07:12:22.596135+0000","last_became_active":"2024-09-13T07:12:20.600877+0000","last_became_peered":"2024-09-13T07:12:20.600877+0000","last_unstale":"2024-09-13T07:12:22.596135+0000","last_undegraded":"2024-09-13T07:12:22.596135+0000","last_fullsized":"2024-09-13T07:12:22.596135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:57.330291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077577500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.742180+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:12:22.742180+0000","last_peered":"2024-09-13T07:12:22.742180+0000","last_clean":"2024-09-13T07:12:22.742180+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:12:22.742180+0000","last_undegraded":"2024-09-13T07:12:22.742180+0000","last_fullsized":"2024-09-13T07:12:22.742180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592787+0000","last_change":"2024-09-13T07:12:22.592787+0000","last_active":"2024-09-13T07:12:22.592787+0000","last_peered":"2024-09-13T07:12:22.592787+0000","last_clean":"2024-09-13T07:12:22.592787+0000","last_became_active":"2024-09-13T07:12:20.600886+0000","last_became_peered":"2024-09-13T07:12:20.600886+0000","last_unstale":"2024-09-13T07:12:22.592787+0000","last_undegraded":"2024-09-13T07:12:22.592787+0000","last_fullsized":"2024-09-13T07:12:22.592787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:18.571233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024777700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:12:26.719 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T07:12:26.719 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T07:12:26.719 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T07:12:26.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T07:12:27.037 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:27.037 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T07:12:27.049 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T07:12:27.050 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T07:12:27.062 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T07:12:27.062 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T07:12:27.062 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T07:12:27.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T07:12:27.366 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T07:12:27.366 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T07:12:27.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:12:27.649 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:12:27.661 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-13T07:12:26.469700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217632,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757088,"statfs":{"total":773094113280,"available":772871258112,"internally_reserved":0,"allocated":1466368,"data_stored":1607262,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000441"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590999+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:12:22.590999+0000","last_peered":"2024-09-13T07:12:22.590999+0000","last_clean":"2024-09-13T07:12:22.590999+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:12:22.590999+0000","last_undegraded":"2024-09-13T07:12:22.590999+0000","last_fullsized":"2024-09-13T07:12:22.590999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590994+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:12:22.590994+0000","last_peered":"2024-09-13T07:12:22.590994+0000","last_clean":"2024-09-13T07:12:22.590994+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:12:22.590994+0000","last_undegraded":"2024-09-13T07:12:22.590994+0000","last_fullsized":"2024-09-13T07:12:22.590994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.863101+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:12:22.863101+0000","last_peered":"2024-09-13T07:12:22.863101+0000","last_clean":"2024-09-13T07:12:22.863101+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:12:22.863101+0000","last_undegraded":"2024-09-13T07:12:22.863101+0000","last_fullsized":"2024-09-13T07:12:22.863101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592265+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:12:22.592265+0000","last_peered":"2024-09-13T07:12:22.592265+0000","last_clean":"2024-09-13T07:12:22.592265+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:12:22.592265+0000","last_undegraded":"2024-09-13T07:12:22.592265+0000","last_fullsized":"2024-09-13T07:12:22.592265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:23.040450+0000","last_change":"2024-09-13T07:12:21.825606+0000","last_active":"2024-09-13T07:12:23.040450+0000","last_peered":"2024-09-13T07:12:23.040450+0000","last_clean":"2024-09-13T07:12:23.040450+0000","last_became_active":"2024-09-13T07:12:21.824606+0000","last_became_peered":"2024-09-13T07:12:21.824606+0000","last_unstale":"2024-09-13T07:12:23.040450+0000","last_undegraded":"2024-09-13T07:12:23.040450+0000","last_fullsized":"2024-09-13T07:12:23.040450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:05.013770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592051+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:12:22.592051+0000","last_peered":"2024-09-13T07:12:22.592051+0000","last_clean":"2024-09-13T07:12:22.592051+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:12:22.592051+0000","last_undegraded":"2024-09-13T07:12:22.592051+0000","last_fullsized":"2024-09-13T07:12:22.592051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.596135+0000","last_change":"2024-09-13T07:12:22.593317+0000","last_active":"2024-09-13T07:12:22.596135+0000","last_peered":"2024-09-13T07:12:22.596135+0000","last_clean":"2024-09-13T07:12:22.596135+0000","last_became_active":"2024-09-13T07:12:20.600877+0000","last_became_peered":"2024-09-13T07:12:20.600877+0000","last_unstale":"2024-09-13T07:12:22.596135+0000","last_undegraded":"2024-09-13T07:12:22.596135+0000","last_fullsized":"2024-09-13T07:12:22.596135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:57.330291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077577500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.742180+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:12:22.742180+0000","last_peered":"2024-09-13T07:12:22.742180+0000","last_clean":"2024-09-13T07:12:22.742180+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:12:22.742180+0000","last_undegraded":"2024-09-13T07:12:22.742180+0000","last_fullsized":"2024-09-13T07:12:22.742180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592787+0000","last_change":"2024-09-13T07:12:22.592787+0000","last_active":"2024-09-13T07:12:22.592787+0000","last_peered":"2024-09-13T07:12:22.592787+0000","last_clean":"2024-09-13T07:12:22.592787+0000","last_became_active":"2024-09-13T07:12:20.600886+0000","last_became_peered":"2024-09-13T07:12:20.600886+0000","last_unstale":"2024-09-13T07:12:22.592787+0000","last_undegraded":"2024-09-13T07:12:22.592787+0000","last_fullsized":"2024-09-13T07:12:22.592787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:18.571233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024777700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:12:27.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:12:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:27.948 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:12:27.960 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-13T07:12:26.469700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217632,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757088,"statfs":{"total":773094113280,"available":772871258112,"internally_reserved":0,"allocated":1466368,"data_stored":1607262,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000441"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590999+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:12:22.590999+0000","last_peered":"2024-09-13T07:12:22.590999+0000","last_clean":"2024-09-13T07:12:22.590999+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:12:22.590999+0000","last_undegraded":"2024-09-13T07:12:22.590999+0000","last_fullsized":"2024-09-13T07:12:22.590999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590994+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:12:22.590994+0000","last_peered":"2024-09-13T07:12:22.590994+0000","last_clean":"2024-09-13T07:12:22.590994+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:12:22.590994+0000","last_undegraded":"2024-09-13T07:12:22.590994+0000","last_fullsized":"2024-09-13T07:12:22.590994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.863101+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:12:22.863101+0000","last_peered":"2024-09-13T07:12:22.863101+0000","last_clean":"2024-09-13T07:12:22.863101+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:12:22.863101+0000","last_undegraded":"2024-09-13T07:12:22.863101+0000","last_fullsized":"2024-09-13T07:12:22.863101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592265+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:12:22.592265+0000","last_peered":"2024-09-13T07:12:22.592265+0000","last_clean":"2024-09-13T07:12:22.592265+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:12:22.592265+0000","last_undegraded":"2024-09-13T07:12:22.592265+0000","last_fullsized":"2024-09-13T07:12:22.592265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:23.040450+0000","last_change":"2024-09-13T07:12:21.825606+0000","last_active":"2024-09-13T07:12:23.040450+0000","last_peered":"2024-09-13T07:12:23.040450+0000","last_clean":"2024-09-13T07:12:23.040450+0000","last_became_active":"2024-09-13T07:12:21.824606+0000","last_became_peered":"2024-09-13T07:12:21.824606+0000","last_unstale":"2024-09-13T07:12:23.040450+0000","last_undegraded":"2024-09-13T07:12:23.040450+0000","last_fullsized":"2024-09-13T07:12:23.040450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:05.013770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592051+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:12:22.592051+0000","last_peered":"2024-09-13T07:12:22.592051+0000","last_clean":"2024-09-13T07:12:22.592051+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:12:22.592051+0000","last_undegraded":"2024-09-13T07:12:22.592051+0000","last_fullsized":"2024-09-13T07:12:22.592051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.596135+0000","last_change":"2024-09-13T07:12:22.593317+0000","last_active":"2024-09-13T07:12:22.596135+0000","last_peered":"2024-09-13T07:12:22.596135+0000","last_clean":"2024-09-13T07:12:22.596135+0000","last_became_active":"2024-09-13T07:12:20.600877+0000","last_became_peered":"2024-09-13T07:12:20.600877+0000","last_unstale":"2024-09-13T07:12:22.596135+0000","last_undegraded":"2024-09-13T07:12:22.596135+0000","last_fullsized":"2024-09-13T07:12:22.596135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:57.330291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077577500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.742180+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:12:22.742180+0000","last_peered":"2024-09-13T07:12:22.742180+0000","last_clean":"2024-09-13T07:12:22.742180+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:12:22.742180+0000","last_undegraded":"2024-09-13T07:12:22.742180+0000","last_fullsized":"2024-09-13T07:12:22.742180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592787+0000","last_change":"2024-09-13T07:12:22.592787+0000","last_active":"2024-09-13T07:12:22.592787+0000","last_peered":"2024-09-13T07:12:22.592787+0000","last_clean":"2024-09-13T07:12:22.592787+0000","last_became_active":"2024-09-13T07:12:20.600886+0000","last_became_peered":"2024-09-13T07:12:20.600886+0000","last_unstale":"2024-09-13T07:12:22.592787+0000","last_undegraded":"2024-09-13T07:12:22.592787+0000","last_fullsized":"2024-09-13T07:12:22.592787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:18.571233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024777700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:12:27.961 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T07:12:27.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:fsid f667c631-bb9d-47bf-97c6-7217d8893244 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-13T07:12:08.346514+0000 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-13T07:12:28.126802+0000 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 4 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-13T07:12:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client jewel 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release reef 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.00 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 8 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/1709347595,v1:172.21.15.31:6809/1709347595] [v2:172.21.15.31:6810/1709347595,v1:172.21.15.31:6811/1709347595] exists,up c29fb342-d45b-48c1-9694-dd01e2a01325 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6824/3300827638,v1:172.21.15.31:6825/3300827638] [v2:172.21.15.31:6826/3300827638,v1:172.21.15.31:6827/3300827638] exists,up 4b979b5c-a78b-461d-b795-f6f5c25cf652 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/3977233096,v1:172.21.15.31:6801/3977233096] [v2:172.21.15.31:6802/3977233096,v1:172.21.15.31:6803/3977233096] exists,up 11259f54-1134-495f-9281-d5132aff2780 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/3290545828,v1:172.21.15.31:6817/3290545828] [v2:172.21.15.31:6818/3290545828,v1:172.21.15.31:6819/3290545828] exists,up a11be2be-3803-4a23-bbf8-40ea110ccf41 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/3543694628,v1:172.21.15.184:6801/3543694628] [v2:172.21.15.184:6802/3543694628,v1:172.21.15.184:6803/3543694628] exists,up 2b6941d9-29f5-4836-a61c-47ff27a7172e 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/951246042,v1:172.21.15.184:6810/951246042] [v2:172.21.15.184:6812/951246042,v1:172.21.15.184:6814/951246042] exists,up 17e3a02d-28e3-4126-a8f2-c7783e524d62 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/1941674146,v1:172.21.15.184:6825/1941674146] [v2:172.21.15.184:6826/1941674146,v1:172.21.15.184:6827/1941674146] exists,up ddf29f57-7733-42fa-94ed-015a529fb689 2024-09-13T07:12:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6809/322162340,v1:172.21.15.184:6811/322162340] [v2:172.21.15.184:6813/322162340,v1:172.21.15.184:6815/322162340] exists,up d91e2eed-3bfb-4eb0-ac30-18770c106742 2024-09-13T07:12:28.262 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/1709347595,v1:172.21.15.31:6809/1709347595] [v2:172.21.15.31:6810/1709347595,v1:172.21.15.31:6811/1709347595] exists,up c29fb342-d45b-48c1-9694-dd01e2a01325', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6824/3300827638,v1:172.21.15.31:6825/3300827638] [v2:172.21.15.31:6826/3300827638,v1:172.21.15.31:6827/3300827638] exists,up 4b979b5c-a78b-461d-b795-f6f5c25cf652', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/3977233096,v1:172.21.15.31:6801/3977233096] [v2:172.21.15.31:6802/3977233096,v1:172.21.15.31:6803/3977233096] exists,up 11259f54-1134-495f-9281-d5132aff2780', 'osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/3290545828,v1:172.21.15.31:6817/3290545828] [v2:172.21.15.31:6818/3290545828,v1:172.21.15.31:6819/3290545828] exists,up a11be2be-3803-4a23-bbf8-40ea110ccf41', 'osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/3543694628,v1:172.21.15.184:6801/3543694628] [v2:172.21.15.184:6802/3543694628,v1:172.21.15.184:6803/3543694628] exists,up 2b6941d9-29f5-4836-a61c-47ff27a7172e', 'osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/951246042,v1:172.21.15.184:6810/951246042] [v2:172.21.15.184:6812/951246042,v1:172.21.15.184:6814/951246042] exists,up 17e3a02d-28e3-4126-a8f2-c7783e524d62', 'osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/1941674146,v1:172.21.15.184:6825/1941674146] [v2:172.21.15.184:6826/1941674146,v1:172.21.15.184:6827/1941674146] exists,up ddf29f57-7733-42fa-94ed-015a529fb689', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6809/322162340,v1:172.21.15.184:6811/322162340] [v2:172.21.15.184:6813/322162340,v1:172.21.15.184:6815/322162340] exists,up d91e2eed-3bfb-4eb0-ac30-18770c106742'] 2024-09-13T07:12:28.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T07:12:28.364 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:28.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mon.a", 2024-09-13T07:12:28.364 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster": "ceph", 2024-09-13T07:12:28.364 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T07:12:28.364 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket_mode": "", 2024-09-13T07:12:28.364 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T07:12:28.364 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_client_required": "cephx, none", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_cluster_required": "cephx", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_debug": "true", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_required": "cephx", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_supported": "cephx", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio": "true", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_reap_max": "16", 2024-09-13T07:12:28.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_async_discard": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_block_size": "4096", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_enable_discard": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry": "3", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash": "0", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_hipri": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T07:12:28.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_type": "", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_allocator": "hybrid", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_buffered_io": "true", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T07:12:28.367 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery": "false", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_sync_write": "false", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T07:12:28.368 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocator": "hybrid", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_create": "true", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_create": "false", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_path": "", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_size": "0", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_path": "", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T07:12:28.369 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_size": "107374182400", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_create": "false", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_path": "", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs": "true", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune": "true", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T07:12:28.370 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size": "0", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_type": "2q", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_clone_cow": "true", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T07:12:28.371 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_mode": "none", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_freelist": "false", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T07:12:28.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T07:12:28.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fail_eio": "false", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T07:12:28.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T07:12:28.375 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size": "0", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T07:12:28.376 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_mem": "512", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T07:12:28.377 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_tracing": "false", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T07:12:28.378 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T07:12:28.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_version": "2", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_signatures": "false", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_version": "2", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_signatures": "false", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_version": "2", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_sign_messages": "true", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "chdir": "", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_acl_type": "", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_asio_thread_count": "2", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_mid": "0.750000", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_size": "16384", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_caps_release_delay": "5", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_check_pool_perm": "true", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_force_sync_read": "false", 2024-09-13T07:12:28.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_getattr_caps": "false", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_features": "", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_remount": "false", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_dirsize_rbytes": "true", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_force_lazyio": "false", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_fs": "", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_release_failure": "false", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_inline_size": "4096", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mds_namespace": "", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_metadata": "", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_gid": "-1", 2024-09-13T07:12:28.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_timeout": "300", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_uid": "-1", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mountpoint": "/", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_notify_timeout": "10", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc": "true", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_objects": "1000", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_size": "209715200", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_permissions": "true", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota": "true", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota_df": "true", 2024-09-13T07:12:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_bytes": "0", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_periods": "4", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_min": "131072", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_reconnect_stale": "false", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_shutdown_timeout": "30", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_snapdir": ".snap", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_tick_interval": "1", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_trace": "", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_faked_inos": "false", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_random_mds": "false", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog": "false", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_port": "12201", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_monitors": "default=true", 2024-09-13T07:12:28.383 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog": "false", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_level": "info", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "-", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network": "", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network_interface": "", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_isal": "false", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_level": "5", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zstd_level": "1", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T07:12:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_smp": "1", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location": "", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook": "", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook_timeout": "10", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "daemonize": "false", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "daos_pool": "tank", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_alienstore": "0/5", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok": "1/5", 2024-09-13T07:12:28.385 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok_assert_abort": "false", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_auth": "20/20", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bdev": "1/3", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluefs": "1/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluestore": "1/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_buffer": "0/1", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephsqlite": "0/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_client": "0/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_compressor": "1/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_context": "0/1", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crush": "1/1", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crypto": "1/5", 2024-09-13T07:12:28.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cyanstore": "0/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_dpdk": "1/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_eventtrace": "1/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filer": "0/1", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filestore": "1/3", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_finisher": "1/1", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_fuse": "1/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_javaclient": "1/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journal": "1/3", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journaler": "0/5", 2024-09-13T07:12:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_kstore": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_leveldb": "4/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_lockdep": "0/1", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mclock": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_balancer": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_locker": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_migrator": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_memstore": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgr": "2/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgrc": "1/5", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mon": "20/20", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_monc": "0/10", 2024-09-13T07:12:28.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ms": "1/1", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_none": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objclass": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objectcacher": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objecter": "0/1", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_optracker": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_osd": "1/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_paxos": "20/20", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_perfcounter": "1/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_prioritycache": "1/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rados": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_replay": "0/5", 2024-09-13T07:12:28.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_refs": "0/0", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_reserver": "1/1", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw": "1/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_access": "1/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_flight": "1/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_sync": "1/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rocksdb": "4/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore": "0/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_backref": "0/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cache": "0/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_device": "0/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_epm": "0/5", 2024-09-13T07:12:28.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_journal": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_lba": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_odata": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_omap": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_onode": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_t": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_tm": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_striper": "0/1", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_test": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_throttle": "1/1", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_timer": "0/1", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_tp": "0/5", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T07:12:28.391 INFO:teuthology.orchestra.run.smithi031.stdout: "device_failure_prediction_mode": "none", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_graylog": "false", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_journald": "false", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_stderr": "true", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_syslog": "false", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event_tracing": "false", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_http_port": "9926", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_prio_limit": "5", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sort_metrics": "true", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_stats_period": "5", 2024-09-13T07:12:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: "fake_statfs_for_testing": "0", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "fatal_signal_handlers": "true", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_purge_ops": "10", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_truncate_ops": "128", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_blackhole": "false", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_snap": "true", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_caller_concurrency": "10", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_omap_check": "false", 2024-09-13T07:12:28.393 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_verify_split": "false", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_dump_file": "", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fadvise": "true", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fail_eio": "true", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_size": "128", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap": "false", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T07:12:28.394 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_inject_stall": "0", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_parallel": "false", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_trailing": "false", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_writeahead": "false", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_kill_at": "0", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T07:12:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_merge_threshold": "-10", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_odsync_write": "false", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend_path": "", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T07:12:28.396 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_threads": "2", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_punch_hole": "false", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_ops": "50", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_seek_data_hole": "false", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc": "false", 2024-09-13T07:12:28.397 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_splice": "false", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_multiple": "2", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_rand_factor": "20", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_update_to": "1000", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T07:12:28.398 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_xfs_extsize": "false", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_zfs_snap": "false", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fio_dir": "/tmp/fio", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "f667c631-bb9d-47bf-97c6-7217d8893244", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_allow_other": "true", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_big_writes": "true", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_debug": "false", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_default_permissions": "false", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_disable_pagecache": "false", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_max_write": "0", 2024-09-13T07:12:28.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_multithreaded": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_require_active_mds": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_set_user_groups": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_move": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_read": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_write": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_target_name": "ceph", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_file": "", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_inject_failure": "0", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_interval": "5", 2024-09-13T07:12:28.400 INFO:teuthology.orchestra.run.smithi031.stdout: "host": "smithi031", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T07:12:28.401 INFO:teuthology.orchestra.run.smithi031.stdout: "inject_early_sigterm": "false", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_agent_port": "6799", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_tracing_enable": "false", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_aio": "true", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_align_min_size": "65536", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_align": "true", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_size": "4096", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_dio": "true", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_discard": "false", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_force_aio": "false", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_ignore_corruption": "false", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_entries": "100", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_replay_from": "0", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T07:12:28.402 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_write_header_frequency": "0", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_zero_on_create": "false", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prefetch_periods": "10", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prezero_periods": "5", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_write_head_interval": "15", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "keyfile": "", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_backend": "rocksdb", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T07:12:28.403 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_bytes": "67108864", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_ops": "512", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_onode_map_size": "1024", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_transaction": "false", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_block_size": "65536", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_bloom_size": "0", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_cache_size": "536870912", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T07:12:28.404 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_compression": "false", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_log": "", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_max_open_files": "0", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_paranoid": "false", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "librados_thread_count": "2", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep": "false", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep_force_backtrace": "false", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_coarse_timestamps": "true", 2024-09-13T07:12:28.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T07:12:28.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_flush_on_exit": "false", 2024-09-13T07:12:28.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T07:12:28.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_port": "12201", 2024-09-13T07:12:28.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_new": "1000", 2024-09-13T07:12:28.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_recent": "10000", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stderr_prefix": "", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": "true", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_graylog": "false", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_journald": "false", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_stderr": "false", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_syslog": "false", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_action_on_write_error": "1", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_alternate_name_max": "8192", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_asio_thread_count": "2", 2024-09-13T07:12:28.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_export_pin": "true", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_interval": "10", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max": "-1", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max_until": "-1", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_merge_size": "50", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_mode": "0", 2024-09-13T07:12:28.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_bits": "3", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_size": "10000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_mid": "0.700000", 2024-09-13T07:12:28.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_interval": "1", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_auth_pins": "false", 2024-09-13T07:12:28.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_frag": "false", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_scatterstat": "false", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_subtrees": "false", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_default_dir_hash": "2", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_defer_session_stale": "true", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_entries": "0", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_prefetch": "true", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T07:12:28.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_early_reply": "true", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enable_op_tracker": "true", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enforce_unique_name": "true", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T07:12:28.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_health_dummy": "false", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T07:12:28.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_join_fs": "", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_journal_format": "1", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_create_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_export_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_import_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_link_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_openc_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_rename_at": "0", 2024-09-13T07:12:28.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_events": "-1", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_segments": "128", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_pause": "false", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_segment_size": "0", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_requests": "100000", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_export_size": "20971520", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_file_recover": "32", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_files": "64", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops": "8192", 2024-09-13T07:12:28.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_metrics_update_interval": "2", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_per_client": "100", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_numa_node": "-1", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_duration": "600", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_size": "20", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T07:12:28.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_log_threshold": "5", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_grace": "15", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_interval": "5", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_caps": "30000", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_interval": "1.000000", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T07:12:28.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_gid": "0", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_uid": "0", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_shutdown_check": "0", 2024-09-13T07:12:28.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_skip_ino": "0", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_min_uid": "0", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_rstat": "false", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_symlink_recovery": "true", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_exports": "0", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_fragments": "0", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_tick_interval": "5.000000", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_valgrind_exit": "false", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_backtrace": "1", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_scatter": "false", 2024-09-13T07:12:28.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_sessions": "false", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool_debug": "false", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_set": "false", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_size": "65536", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_bytes": "134217728", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_messages": "512", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_disabled_modules": "", 2024-09-13T07:12:28.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_messages": "128", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_messages": "128", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_messages": "8192", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_pool": "true", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_standby_modules": "true", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_period": "5", 2024-09-13T07:12:28.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_threshold": "5", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_tick_period": "2", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_delete": "true", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_bytes": "104857600", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T07:12:28.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T07:12:28.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_start": "false", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_trim": "true", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cpu_threads": "4", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T07:12:28.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_crit": "5", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_warn": "5", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_json": "false", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_extra_checks": "false", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T07:12:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_default_strategy": "1", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_timeout": "5.000000", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_enable_op_tracker": "true", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fake_pool_delete": "false", 2024-09-13T07:12:28.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_force_quorum_join": "false", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_log_update_period": "5", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_max_detail": "50", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog": "true", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host": "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.184", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host_override": "", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_initial_members": "", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T07:12:28.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease": "5.000000", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_full_interval": "50", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max": "10000", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max_summary": "50", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_epochs": "500", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_osd": "10000", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T07:12:28.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_autotune": "true", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_target": "2147483648", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_digest_period": "5", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T07:12:28.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_complaint_time": "30", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_duration": "600", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_size": "20", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_log_threshold": "5", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T07:12:28.432 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size": "500", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T07:12:28.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_report_timeout": "900", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T07:12:28.434 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T07:12:28.435 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_osds": "4", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_interval": "86400", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_max_keys": "100", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_timeout": "300", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_session_timeout": "300", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_smart_report_timeout": "5", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T07:12:28.436 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_size": "4", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_debug": "false", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T07:12:28.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_tick_interval": "5", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_older_version": "true", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T07:12:28.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "monmap": "", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T07:12:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_tracing_enabled": "false", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_op_threads": "3", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_cm": "false", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_device_name": "", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T07:12:28.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_sl": "3", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_type": "ib", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_reap_threshold": "5", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_before_connect": "false", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv4": "true", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv6": "false", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr1": "true", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr2": "true", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_max": "7568", 2024-09-13T07:12:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_min": "6800", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_count": "3", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_delay": "5", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_client": "false", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mds": "false", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mgr": "false", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mon": "false", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_osd": "false", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_client_mode": "crc secure", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_type": "", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_compress_secure": "false", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T07:12:28.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_data": "true", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_header": "true", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bug": "true", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_old_message": "true", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T07:12:28.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hugepages": "", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_lro": "true", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_pmd": "", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_port_id": "0", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_on_send": "false", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T07:12:28.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_type": "", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_network_congestion": "0", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_accept_failures": "4", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_backoff": "15.000000", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_mode": "none", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T07:12:28.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_min_cost": "65536", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_public_type": "", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_service_mode": "crc secure", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_nodelay": "true", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_type": "async+posix", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "no_config_file": "false", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_ops": "1024", 2024-09-13T07:12:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_timeout": "10.000000", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_blackhole": "false", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "openssl_engine_opts": "", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_ops": "4", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_slop": "0.020000", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T07:12:28.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_asio_thread_count": "2", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_max": "512", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_min": "64", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_peering": "false", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_beacon_report_interval": "300", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_duration": "30", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T07:12:28.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blkin_trace_all": "false", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_update_on_start": "true", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_cap": "256", 2024-09-13T07:12:28.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_op_priority": "63", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_watch_timeout": "30", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_command_max_records": "256", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_compact_on_start": "false", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_on_start": "true", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T07:12:28.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_op_order": "false", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T07:12:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_shutdown": "false", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T07:12:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_notify_timeout": "30", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_enable_op_tracker": "true", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T07:12:28.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_info": "true", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown": "true", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_function_tracing": "false", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_grace": "20", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_interval": "6", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_stale": "600", 2024-09-13T07:12:28.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_size": "5120", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_kill_backfill_at": "0", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_cache_size": "50", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_dedup": "true", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max": "40", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T07:12:28.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_name_len": "100", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_size": "0", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_backfills": "1", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_count": "5", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_period": "600", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_name_len": "2048", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_size": "134217728", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T07:12:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pgls": "1024", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_cost": "8388608", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_objects": "10", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_scrubs": "3", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_size": "90", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T07:12:28.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_profile": "balanced", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_base": "805306368", 2024-09-13T07:12:28.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target": "4294967296", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_autotune": "false", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_recovery_priority": "0", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_interval": "5", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T07:12:28.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_cache_shards": "32", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_node": "-1", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objecter_finishers": "1", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore": "bluestore", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_fuse": "false", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_tracing": "false", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_duration": "600", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_size": "20", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T07:12:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_log_threshold": "5", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards": "0", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T07:12:28.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_timeout": "15", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_open_classes_on_start": "true", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_os_flags": "0", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_peering_op_priority": "255", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_priority": "5", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T07:12:28.462 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crimson": "false", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T07:12:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flags": "0", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_min_size": "0", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_size": "2", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_type": "replicated", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T07:12:28.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_cost": "20971520", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active": "0", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_priority": "3", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_priority": "5", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T07:12:28.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_repair_during_recovery": "false", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T07:12:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_cost": "52428800", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_hour": "0", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_priority": "5", 2024-09-13T07:12:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_skip_data_digest": "false", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_smart_report_timeout": "5", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_priority": "5", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T07:12:28.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_transaction_size": "30", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T07:12:28.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tracing": "false", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_use_stale_snap": "false", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_kill_at": "0", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_max_join_drift": "10", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min": "500", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min_wait": "0.050000", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max": "500", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_min": "250", 2024-09-13T07:12:28.470 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_stash_full_interval": "25", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_max": "500", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_min": "250", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "perf": "true", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "-", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrv": "", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "public_bind_addr": "-", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network": "", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network_interface": "", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_enabled": "false", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T07:12:28.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_mon_op_timeout": "0", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_osd_op_timeout": "0", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_tracing": "false", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_atime_update_interval": "60", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache": "true", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T07:12:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_policy": "writearound", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_size": "33554432", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_compression_hint": "none", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_clone_format": "auto", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_data_pool": "", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_features": "61", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_format": "2", 2024-09-13T07:12:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_map_options": "", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_order": "22", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_pool": "rbd", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_count": "0", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler": "simple", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T07:12:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_order": "24", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_pool": "", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_splay_width": "4", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T07:12:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T07:12:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T07:12:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_threads": "1", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_plugins": "", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T07:12:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T07:12:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_tracing": "false", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_names": "true", 2024-09-13T07:12:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_pool": "true", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_base_url": "", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_log_level": "", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_admin_entry": "admin", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_backend_store": "rados", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_barbican_url": "", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_beast_enable_async": "true", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T07:12:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_enabled": "true", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_config_store": "rados", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_content_length_compat": "false", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cross_domain_policy": "", 2024-09-13T07:12:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T07:12:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T07:12:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T07:12:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_window": "30", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T07:12:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T07:12:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_name": "", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_s3website_name": "", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T07:12:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_ops_log": "false", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_static_website": "false", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_usage_log": "false", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_expose_bucket": "false", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_extended_http_attrs": "", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_filter": "none", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T07:12:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_objs": "32", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_init_timeout": "300", 2024-09-13T07:12:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_project": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_user": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_api_version": "2", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T07:12:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_url": "", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T07:12:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_objs": "32", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_worker": "3", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T07:12:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_http_headers": "", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_size": "0", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T07:12:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_listing_results": "1000", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_notify_retries": "10", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T07:12:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T07:12:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_control_oids": "8", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_numa_node": "-1", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_tracing": "false", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_token": "", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_url": "", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T07:12:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_rados": "false", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_continue": "true", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T07:12:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_tracing": "false", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm": "", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_id": "", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region": "", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_request_uri": "", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T07:12:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_resolve_cname": "false", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_run_sync_thread": "true", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T07:12:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_script_uri": "", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_service_provider_name": "", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_id": "", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_secret": "", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_entry": "sts", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_key": "sts", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T07:12:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_url": "", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_custom_header": "", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_need_stats": "true", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_tenant_name": "", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url": "", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_lease_period": "120", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T07:12:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_thread_pool_size": "512", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_comment": "", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_createby": "", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_encoding": "", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_flag": "false", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_origin": "", 2024-09-13T07:12:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_tracker": "", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_shards": "32", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_use_opa_authz": "false", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T07:12:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_unique_email": "true", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_verify_ssl": "true", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone": "", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_id": "", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup": "", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_id": "", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T07:12:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_block_size": "4096", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T07:12:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_partition_filters": "false", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_perf": "false", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "run_dir": "/var/run/ceph", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_block_create": "true", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T07:12:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_device_size": "53687091200", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_main_device_type": "SSD", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_segment_size": "67108864", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "set_keepcaps": "false", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "setgroup": "", 2024-09-13T07:12:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser": "", 2024-09-13T07:12:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser_match_path": "", 2024-09-13T07:12:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T07:12:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "thp": "false", 2024-09-13T07:12:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_default_timeout": "60", 2024-09-13T07:12:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T07:12:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "throttler_perf_counter": "true" 2024-09-13T07:12:28.509 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:28.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T07:12:28.627 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-13T07:12:28.627 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:12:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-13T07:12:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:12:28.646 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-13T07:12:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:12:28.657 DEBUG:teuthology.orchestra.run.smithi031:> type ceph-objectstore-tool 2024-09-13T07:12:28.672 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T07:12:28.672 DEBUG:teuthology.orchestra.run.smithi184:> type ceph-objectstore-tool 2024-09-13T07:12:28.690 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T07:12:28.690 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T07:12:28.700 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T07:12:28.700 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T07:12:28.700 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:12:28.701 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:12:28.701 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:12:28.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:12:28.701 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T07:12:28.702 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T07:12:28.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:12:28.702 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T07:12:28.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:28.703 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T07:12:28.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:12:28.703 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-13T07:12:28.714 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:28.714+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:28.750 INFO:teuthology.orchestra.run.smithi031.stdout:mapped writes DISABLED 2024-09-13T07:12:28.750 INFO:teuthology.orchestra.run.smithi031.stdout:mapped reads DISABLED 2024-09-13T07:12:28.750 INFO:teuthology.orchestra.run.smithi031.stdout:Seed set to 5901 2024-09-13T07:12:28.803 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:28.804+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:28.875 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:28.876 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:28.876 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:28.876 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:28.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:28.895 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:28.904 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:28.905+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:28.913 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:28.913 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:28.930 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:28.946 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:28.947 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:28.964 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:28.964 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:28.981 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:28.998 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:28.998 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:29.004 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:29.004 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:29.004 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:29.005+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:29.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:29.015 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:12:29.016 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:12:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 18, 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "f667c631-bb9d-47bf-97c6-7217d8893244", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T07:12:08.346514+0000", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T07:12:28.126802+0000", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T07:12:18.566379+0000", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T07:12:10.626165+0000", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T07:12:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 2, 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T07:12:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:19.268005+0000", 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:12:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:12:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 17, 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:12:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:12:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:12:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:20.513150+0000", 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:12:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:12:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:12:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:12:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T07:12:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:12:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c29fb342-d45b-48c1-9694-dd01e2a01325", 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/1709347595", 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/1709347595", 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/1709347595", 2024-09-13T07:12:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/1709347595", 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4b979b5c-a78b-461d-b795-f6f5c25cf652", 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 14, 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3300827638", 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3300827638", 2024-09-13T07:12:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3300827638", 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3300827638", 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "11259f54-1134-495f-9281-d5132aff2780", 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:12:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3977233096", 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3977233096", 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3977233096", 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3977233096", 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "a11be2be-3803-4a23-bbf8-40ea110ccf41", 2024-09-13T07:12:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 14, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/3290545828", 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/3290545828", 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/3290545828", 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/3290545828", 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:12:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2b6941d9-29f5-4836-a61c-47ff27a7172e", 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 14, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6800", 2024-09-13T07:12:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6801", 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6802", 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6803", 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6806", 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6807", 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6804", 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6805", 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6801/3543694628", 2024-09-13T07:12:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6803/3543694628", 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6807/3543694628", 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6805/3543694628", 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "17e3a02d-28e3-4126-a8f2-c7783e524d62", 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 14, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6808", 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6810", 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6812", 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6814", 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6820", 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6821", 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6816", 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6818", 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6810/951246042", 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6814/951246042", 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6821/951246042", 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6818/951246042", 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ddf29f57-7733-42fa-94ed-015a529fb689", 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 14, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6824", 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6825", 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6826", 2024-09-13T07:12:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6827", 2024-09-13T07:12:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6830", 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6831", 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6828", 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6829", 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6825/1941674146", 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6827/1941674146", 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6831/1941674146", 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6829/1941674146", 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:12:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d91e2eed-3bfb-4eb0-ac30-18770c106742", 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 15, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6809", 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6811", 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6813", 2024-09-13T07:12:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6815", 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6822", 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6823", 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6817", 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6819", 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6811/322162340", 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6815/322162340", 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6823/322162340", 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6819/322162340", 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.805494+0000", 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:12:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.817444+0000", 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.712083+0000", 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.739958+0000", 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.720767+0000", 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.715944+0000", 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.788853+0000", 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.809325+0000", 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T07:12:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T07:12:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [ 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 2, 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T07:12:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T07:12:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T07:12:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:12:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:12:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T07:12:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:29.141 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:29.143 INFO:tasks.thrashosds.thrasher:j is {'epoch': 18, 'fsid': 'f667c631-bb9d-47bf-97c6-7217d8893244', 'created': '2024-09-13T07:12:08.346514+0000', 'modified': '2024-09-13T07:12:28.126802+0000', 'last_up_change': '2024-09-13T07:12:18.566379+0000', 'last_in_change': '2024-09-13T07:12:10.626165+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:12:19.268005+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:12:20.513150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c29fb342-d45b-48c1-9694-dd01e2a01325', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 1709347595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 1709347595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 1709347595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 1709347595}]}, 'public_addr': '172.21.15.31:6809/1709347595', 'cluster_addr': '172.21.15.31:6811/1709347595', 'heartbeat_back_addr': '172.21.15.31:6815/1709347595', 'heartbeat_front_addr': '172.21.15.31:6813/1709347595', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4b979b5c-a78b-461d-b795-f6f5c25cf652', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3300827638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3300827638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3300827638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3300827638}]}, 'public_addr': '172.21.15.31:6825/3300827638', 'cluster_addr': '172.21.15.31:6827/3300827638', 'heartbeat_back_addr': '172.21.15.31:6831/3300827638', 'heartbeat_front_addr': '172.21.15.31:6829/3300827638', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '11259f54-1134-495f-9281-d5132aff2780', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3977233096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3977233096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3977233096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3977233096}]}, 'public_addr': '172.21.15.31:6801/3977233096', 'cluster_addr': '172.21.15.31:6803/3977233096', 'heartbeat_back_addr': '172.21.15.31:6807/3977233096', 'heartbeat_front_addr': '172.21.15.31:6805/3977233096', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a11be2be-3803-4a23-bbf8-40ea110ccf41', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 3290545828}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 3290545828}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 3290545828}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 3290545828}]}, 'public_addr': '172.21.15.31:6817/3290545828', 'cluster_addr': '172.21.15.31:6819/3290545828', 'heartbeat_back_addr': '172.21.15.31:6823/3290545828', 'heartbeat_front_addr': '172.21.15.31:6821/3290545828', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2b6941d9-29f5-4836-a61c-47ff27a7172e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 3543694628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 3543694628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 3543694628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 3543694628}]}, 'public_addr': '172.21.15.184:6801/3543694628', 'cluster_addr': '172.21.15.184:6803/3543694628', 'heartbeat_back_addr': '172.21.15.184:6807/3543694628', 'heartbeat_front_addr': '172.21.15.184:6805/3543694628', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '17e3a02d-28e3-4126-a8f2-c7783e524d62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6810', 'nonce': 951246042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6814', 'nonce': 951246042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 951246042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6818', 'nonce': 951246042}]}, 'public_addr': '172.21.15.184:6810/951246042', 'cluster_addr': '172.21.15.184:6814/951246042', 'heartbeat_back_addr': '172.21.15.184:6821/951246042', 'heartbeat_front_addr': '172.21.15.184:6818/951246042', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ddf29f57-7733-42fa-94ed-015a529fb689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 1941674146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 1941674146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 1941674146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 1941674146}]}, 'public_addr': '172.21.15.184:6825/1941674146', 'cluster_addr': '172.21.15.184:6827/1941674146', 'heartbeat_back_addr': '172.21.15.184:6831/1941674146', 'heartbeat_front_addr': '172.21.15.184:6829/1941674146', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd91e2eed-3bfb-4eb0-ac30-18770c106742', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6809', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 322162340}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6813', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 322162340}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 322162340}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6817', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 322162340}]}, 'public_addr': '172.21.15.184:6811/322162340', 'cluster_addr': '172.21.15.184:6815/322162340', 'heartbeat_back_addr': '172.21.15.184:6823/322162340', 'heartbeat_front_addr': '172.21.15.184:6819/322162340', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.805494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.817444+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.712083+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.739958+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.720767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.715944+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.788853+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.809325+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 1, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:12:29.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:12:29.144 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:29.144 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:29.144 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:12:29.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:29.146 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:29.143+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:29.244 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:29.244+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:29.345 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:29.345+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:29.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:29.445 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:29.446+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:29.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:12:29.484 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:12:29.497 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-13T07:12:29.134222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217632,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757088,"statfs":{"total":773094113280,"available":772871258112,"internally_reserved":0,"allocated":1466368,"data_stored":1607262,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.664963"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590999+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:12:22.590999+0000","last_peered":"2024-09-13T07:12:22.590999+0000","last_clean":"2024-09-13T07:12:22.590999+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:12:22.590999+0000","last_undegraded":"2024-09-13T07:12:22.590999+0000","last_fullsized":"2024-09-13T07:12:22.590999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.590994+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:12:22.590994+0000","last_peered":"2024-09-13T07:12:22.590994+0000","last_clean":"2024-09-13T07:12:22.590994+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:12:22.590994+0000","last_undegraded":"2024-09-13T07:12:22.590994+0000","last_fullsized":"2024-09-13T07:12:22.590994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.863101+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:12:22.863101+0000","last_peered":"2024-09-13T07:12:22.863101+0000","last_clean":"2024-09-13T07:12:22.863101+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:12:22.863101+0000","last_undegraded":"2024-09-13T07:12:22.863101+0000","last_fullsized":"2024-09-13T07:12:22.863101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592265+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:12:22.592265+0000","last_peered":"2024-09-13T07:12:22.592265+0000","last_clean":"2024-09-13T07:12:22.592265+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:12:22.592265+0000","last_undegraded":"2024-09-13T07:12:22.592265+0000","last_fullsized":"2024-09-13T07:12:22.592265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:23.040450+0000","last_change":"2024-09-13T07:12:21.825606+0000","last_active":"2024-09-13T07:12:23.040450+0000","last_peered":"2024-09-13T07:12:23.040450+0000","last_clean":"2024-09-13T07:12:23.040450+0000","last_became_active":"2024-09-13T07:12:21.824606+0000","last_became_peered":"2024-09-13T07:12:21.824606+0000","last_unstale":"2024-09-13T07:12:23.040450+0000","last_undegraded":"2024-09-13T07:12:23.040450+0000","last_fullsized":"2024-09-13T07:12:23.040450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:05.013770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592051+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:12:22.592051+0000","last_peered":"2024-09-13T07:12:22.592051+0000","last_clean":"2024-09-13T07:12:22.592051+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:12:22.592051+0000","last_undegraded":"2024-09-13T07:12:22.592051+0000","last_fullsized":"2024-09-13T07:12:22.592051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.596135+0000","last_change":"2024-09-13T07:12:22.593317+0000","last_active":"2024-09-13T07:12:22.596135+0000","last_peered":"2024-09-13T07:12:22.596135+0000","last_clean":"2024-09-13T07:12:22.596135+0000","last_became_active":"2024-09-13T07:12:20.600877+0000","last_became_peered":"2024-09-13T07:12:20.600877+0000","last_unstale":"2024-09-13T07:12:22.596135+0000","last_undegraded":"2024-09-13T07:12:22.596135+0000","last_fullsized":"2024-09-13T07:12:22.596135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:17:57.330291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077577500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.742180+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:12:22.742180+0000","last_peered":"2024-09-13T07:12:22.742180+0000","last_clean":"2024-09-13T07:12:22.742180+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:12:22.742180+0000","last_undegraded":"2024-09-13T07:12:22.742180+0000","last_fullsized":"2024-09-13T07:12:22.742180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T07:12:22.592787+0000","last_change":"2024-09-13T07:12:22.592787+0000","last_active":"2024-09-13T07:12:22.592787+0000","last_peered":"2024-09-13T07:12:22.592787+0000","last_clean":"2024-09-13T07:12:22.592787+0000","last_became_active":"2024-09-13T07:12:20.600886+0000","last_became_peered":"2024-09-13T07:12:20.600886+0000","last_unstale":"2024-09-13T07:12:22.592787+0000","last_undegraded":"2024-09-13T07:12:22.592787+0000","last_fullsized":"2024-09-13T07:12:22.592787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:18.571233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024777700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T07:12:29.128481+0000","last_change":"2024-09-13T07:12:29.128481+0000","last_active":"2024-09-13T07:12:29.128481+0000","last_peered":"2024-09-13T07:12:29.128481+0000","last_clean":"2024-09-13T07:12:29.128481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:12:29.128481+0000","last_undegraded":"2024-09-13T07:12:29.128481+0000","last_fullsized":"2024-09-13T07:12:29.128481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":53330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:12:29.499 INFO:tasks.thrashosds.thrasher:No pools; doing nothing 2024-09-13T07:12:29.546 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:29.547+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:29.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:29.664 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:29.665+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:29.727 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:29.728 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:29.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:29.777 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:29.778+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:29.848 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:29.849+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:29.882 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:29.883 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:29.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:29.949 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:29.950+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:30.038 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.038 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:30.050 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:30.051+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:30.151 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:30.152+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:30.163 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.163 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.173 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:30.257 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:30.258+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:30.319 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.319 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:30.353 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:30.354+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:30.454 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:30.455+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:30.476 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.476 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.485 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:30.554 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:30.555+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:30.598 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.598 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.607 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:30.655 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:30.656+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:30.710 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.710 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.718 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:30.756 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:30.757+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:30.820 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.820 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.829 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:30.873 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:30.874+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:30.931 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:30.931 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:30.939 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:30.980 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:30.981+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:31.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:12:31.073 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:31.074+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:31.080 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:31.080 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:31.089 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:31.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:12:31.169 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:31.170 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:31.170 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:31.170+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:31.187 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:31.193 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:31.193 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:31.201 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:31.203 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:31.203 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:31.260 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:31.261+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:31.304 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:31.305 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:31.313 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:31.397 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:31.398+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:31.421 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:31.421 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:31.456 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:31.456 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:31.457 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:31.461 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:31.462+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:31.466 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:31.472 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:31.489 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:31.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:12:31.506 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:12:31.562 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:31.562+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:31.609 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4257.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.017234076999999, 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036526549999999999, 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594197+0000", 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594211+0000", 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.399e-05 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594220+0000", 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2359999999999993e-06 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594231+0000", 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0293e-05 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594280+0000", 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9261000000000002e-05 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594301+0000", 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0744999999999999e-05 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597812+0000", 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035110950000000001 2024-09-13T07:12:31.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597850+0000", 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.614 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:31.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:12:31.662 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:31.663+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:31.722 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:31.722 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:31.722 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:31.722 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:31.731 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:12:31.763 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:31.764+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:31.832 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:31.832 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:31.832 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:31.832 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:31.832 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:31.840 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:12:31.864 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:31.865+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:31.944 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:31.944 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4257.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.584615+0000", 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.3590655, 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072263029999999999, 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4257", 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/806497074", 2024-09-13T07:12:31.945 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 5 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584615+0000", 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584615+0000", 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584614+0000", 2024-09-13T07:12:31.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584624+0000", 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9979999999999998e-06 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584639+0000", 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5234e-05 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584648+0000", 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0990000000000005e-06 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584698+0000", 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0436999999999998e-05 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.584708+0000", 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3680000000000008e-06 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.587115+0000", 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024071079999999998 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.587230+0000", 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000114663 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.587389+0000", 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0001596 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.588758+0000", 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013684120000000001 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.591750+0000", 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002992579 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.591792+0000", 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1916000000000003e-05 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.591842+0000", 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9508000000000001e-05 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.592431+0000", 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.351249451999999, 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.23577988, 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.951 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592431+0000", 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592431+0000", 2024-09-13T07:12:31.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592430+0000", 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592443+0000", 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2893e-05 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592453+0000", 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0022e-05 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592462+0000", 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8740000000000001e-06 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592512+0000", 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9713999999999999e-05 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592519+0000", 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7300000000000005e-06 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.824875+0000", 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.23235576399999999 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.825012+0000", 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013701499999999999 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.825213+0000", 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020039600000000001 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.826125+0000", 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00091261399999999996 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.828120+0000", 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019948209999999999 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.828170+0000", 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9599999999999999e-05 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.828211+0000", 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1557999999999998e-05 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:32 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866253+0000", 2024-09-13T07:12:31.957 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.077428205, 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092179089999999998, 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 32 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866253+0000", 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866253+0000", 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866252+0000", 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866303+0000", 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0766000000000001e-05 2024-09-13T07:12:31.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866311+0000", 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8319999999999995e-06 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866317+0000", 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9120000000000003e-06 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.867149+0000", 2024-09-13T07:12:31.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00083135399999999997 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.867202+0000", 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3463e-05 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.867291+0000", 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8993999999999995e-05 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869566+0000", 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002275123 2024-09-13T07:12:31.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875409+0000", 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058425789999999997 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875445+0000", 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6049e-05 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875470+0000", 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5675999999999999e-05 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.962 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:33 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866331+0000", 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.077349745999999, 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091850950000000008, 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 33 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866331+0000", 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866331+0000", 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866330+0000", 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866335+0000", 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.454e-06 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866342+0000", 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5410000000000003e-06 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866348+0000", 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8309999999999997e-06 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.867704+0000", 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001356154 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.867754+0000", 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9954999999999999e-05 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.867842+0000", 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7202999999999998e-05 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.870028+0000", 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021866939999999999 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875475+0000", 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054467960000000003 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875499+0000", 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3621e-05 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875516+0000", 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7459999999999999e-05 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:34 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.07732556, 2024-09-13T07:12:31.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092232949999999994, 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 34 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866359+0000", 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5569999999999997e-06 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866366+0000", 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9079999999999998e-06 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866381+0000", 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5231999999999999e-05 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868233+0000", 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001851204 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868283+0000", 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0065999999999998e-05 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868365+0000", 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2354999999999993e-05 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.870405+0000", 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00203981 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875520+0000", 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005114749 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875544+0000", 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4709e-05 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875579+0000", 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4139e-05 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.973 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:35 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866391+0000", 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.077290164000001, 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092673200000000008, 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 35 2024-09-13T07:12:31.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866391+0000", 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866391+0000", 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866390+0000", 2024-09-13T07:12:31.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866467+0000", 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6987e-05 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866476+0000", 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7690000000000002e-06 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866482+0000", 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7780000000000004e-06 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868635+0000", 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002153313 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868697+0000", 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1642000000000004e-05 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868784+0000", 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7052000000000006e-05 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.871584+0000", 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028000019999999998 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875588+0000", 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040048439999999996 2024-09-13T07:12:31.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875626+0000", 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7209000000000002e-05 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875658+0000", 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2255000000000002e-05 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:36 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.979 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866490+0000", 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.077191167000001, 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092203830000000004, 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866490+0000", 2024-09-13T07:12:31.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866490+0000", 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866489+0000", 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866494+0000", 2024-09-13T07:12:31.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.843e-06 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866501+0000", 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866506+0000", 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869175+0000", 2024-09-13T07:12:31.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026688409999999999 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869225+0000", 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0080000000000002e-05 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869316+0000", 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1478999999999996e-05 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.871620+0000", 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023035299999999998 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875662+0000", 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004041958 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875688+0000", 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5972000000000001e-05 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875710+0000", 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2257000000000001e-05 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:37 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.077169594000001, 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092567640000000007, 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.985 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 37 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866515+0000", 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4730000000000002e-06 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866525+0000", 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9289999999999993e-06 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866530+0000", 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869598+0000", 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030676240000000001 2024-09-13T07:12:31.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869675+0000", 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6878000000000005e-05 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869771+0000", 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6249000000000006e-05 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.871648+0000", 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018768509999999999 2024-09-13T07:12:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875715+0000", 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040667530000000002 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875747+0000", 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2759000000000003e-05 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875768+0000", 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0517999999999999e-05 2024-09-13T07:12:31.990 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:88 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.928561+0000", 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 10.015120074, 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092238470000000003, 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.991 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928561+0000", 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928561+0000", 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928560+0000", 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928565+0000", 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3199999999999999e-06 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928572+0000", 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9879999999999998e-06 2024-09-13T07:12:31.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928577+0000", 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6539999999999998e-06 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.932060+0000", 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034830500000000001 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.932131+0000", 2024-09-13T07:12:31.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0581999999999996e-05 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.932209+0000", 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8356999999999995e-05 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.933087+0000", 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087820599999999997 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:31.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.937733+0000", 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046452459999999996 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.937762+0000", 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9271000000000001e-05 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.937785+0000", 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2728e-05 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087833+0000", 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8558482089999995, 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097462420000000004, 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 112 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:31.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087833+0000", 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087833+0000", 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087832+0000", 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:31.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087872+0000", 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9971000000000001e-05 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087885+0000", 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.326e-05 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087891+0000", 2024-09-13T07:12:31.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9070000000000004e-06 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.091686+0000", 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037949720000000002 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.091750+0000", 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4138000000000002e-05 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.091830+0000", 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9639000000000001e-05 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094491+0000", 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002661358 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097523+0000", 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030311090000000001 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097554+0000", 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1501000000000002e-05 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097579+0000", 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4803e-05 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087900+0000", 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8557811609999995, 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097636200000000006, 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 113 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087900+0000", 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087900+0000", 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087899+0000", 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087904+0000", 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087911+0000", 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1829999999999997e-06 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087917+0000", 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8320000000000002e-06 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092280+0000", 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043633170000000002 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092362+0000", 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1786999999999995e-05 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092458+0000", 2024-09-13T07:12:32.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5909999999999995e-05 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094495+0000", 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020376050000000001 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097615+0000", 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00312007 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097642+0000", 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6548999999999999e-05 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097663+0000", 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1433e-05 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.008 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087921+0000", 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8557601449999996, 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.009817961, 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 114 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087921+0000", 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087921+0000", 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087920+0000", 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087924+0000", 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.619e-06 2024-09-13T07:12:32.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087928+0000", 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.608e-06 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087930+0000", 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9490000000000001e-06 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092982+0000", 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050519069999999996 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093025+0000", 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2460999999999999e-05 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093103+0000", 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7645000000000006e-05 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094499+0000", 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013963510000000001 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097695+0000", 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031961619999999998 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097720+0000", 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4737999999999999e-05 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097739+0000", 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8839e-05 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8557433640000003, 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0098704410000000006, 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.014 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 115 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:32.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087978+0000", 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0407999999999997e-05 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087983+0000", 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3519999999999997e-06 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087986+0000", 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.242e-06 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093315+0000", 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005329154 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093366+0000", 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1045999999999999e-05 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093440+0000", 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3283999999999998e-05 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.095499+0000", 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020593959999999998 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097768+0000", 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022686889999999999 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097791+0000", 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2940999999999999e-05 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097808+0000", 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7189000000000001e-05 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8556906889999993, 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0098795619999999997, 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 116 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.020 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087992+0000", 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4399999999999999e-06 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087995+0000", 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1949999999999999e-06 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087998+0000", 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.864e-06 2024-09-13T07:12:32.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093821+0000", 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058222739999999997 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093870+0000", 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093954+0000", 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4053000000000003e-05 2024-09-13T07:12:32.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.095504+0000", 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00155086 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097828+0000", 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00232402 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097851+0000", 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2704000000000001e-05 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097870+0000", 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8476999999999999e-05 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:32.025 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.8556783019999994, 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012635965000000001, 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 117 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088006+0000", 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8670000000000004e-06 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088012+0000", 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1600000000000003e-06 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088015+0000", 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7379999999999999e-06 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094191+0000", 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061757119999999999 2024-09-13T07:12:32.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094240+0000", 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8758999999999999e-05 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094319+0000", 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9567000000000006e-05 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.095719+0000", 2024-09-13T07:12:32.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001399939 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.100525+0000", 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048062790000000001 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.100616+0000", 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0410999999999999e-05 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.100638+0000", 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2734999999999998e-05 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509085+0000", 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4345957620000007, 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0084684089999999997, 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.031 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 137 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509085+0000", 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509085+0000", 2024-09-13T07:12:32.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509085+0000", 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509126+0000", 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1255999999999999e-05 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509133+0000", 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2200000000000003e-06 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509142+0000", 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3839999999999997e-06 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.513995+0000", 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048530730000000003 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.514078+0000", 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2822000000000001e-05 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.514205+0000", 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012782599999999999 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.515651+0000", 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001445329 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517457+0000", 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018057870000000001 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517510+0000", 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3990000000000003e-05 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517553+0000", 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2942000000000003e-05 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509149+0000", 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4345319150000009, 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0085607960000000007, 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 138 2024-09-13T07:12:32.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509149+0000", 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509149+0000", 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509148+0000", 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509153+0000", 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7500000000000003e-06 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509160+0000", 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6980000000000001e-06 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509165+0000", 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.079e-06 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.514737+0000", 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055720910000000004 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.514833+0000", 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6529999999999999e-05 2024-09-13T07:12:32.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.514990+0000", 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015651099999999999 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517567+0000", 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002576886 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517619+0000", 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2369e-05 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517662+0000", 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2760999999999999e-05 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517710+0000", 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7728000000000001e-05 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.434504617, 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010945933, 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 139 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509183+0000", 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3030000000000002e-06 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509198+0000", 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4927e-05 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509204+0000", 2024-09-13T07:12:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1430000000000003e-06 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516097+0000", 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0068931130000000002 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516152+0000", 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5059999999999998e-05 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516244+0000", 2024-09-13T07:12:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2021999999999994e-05 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.518141+0000", 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018971649999999999 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520064+0000", 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001923052 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520097+0000", 2024-09-13T07:12:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2635000000000002e-05 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520122+0000", 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5072999999999999e-05 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4344657040000008, 2024-09-13T07:12:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011295105, 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 140 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509274+0000", 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9419999999999997e-05 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509284+0000", 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0293e-05 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509291+0000", 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4860000000000002e-06 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516552+0000", 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072612099999999997 2024-09-13T07:12:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516632+0000", 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9682e-05 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516764+0000", 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013198800000000001 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.518146+0000", 2024-09-13T07:12:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013828340000000001 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520433+0000", 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022862109999999998 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520476+0000", 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799000000000001e-05 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520510+0000", 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4684999999999997e-05 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509300+0000", 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.4343808189999994, 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014418491, 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 141 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509300+0000", 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509300+0000", 2024-09-13T07:12:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509299+0000", 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509305+0000", 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3090000000000003e-06 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509314+0000", 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7540000000000001e-06 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509320+0000", 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8370000000000002e-06 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517748+0000", 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0084274450000000004 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517803+0000", 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5894999999999999e-05 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517899+0000", 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5884000000000006e-05 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.518705+0000", 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00080585599999999998 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.523539+0000", 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048333739999999997 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.523665+0000", 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000126811 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.523718+0000", 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3084999999999997e-05 2024-09-13T07:12:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:32.060 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:32.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:32.062 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:32.062+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:32.152 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:12:32.161 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:32.162+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:32.187 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:32.187 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:32.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:32.281 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:32.282+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:32.345 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:32.345 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:32.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:32.362 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:32.363+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:32.463 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:32.464+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4257.0:5 1.2 e16/14 1:5cea7035:::rbd_info:head v 16'1, mlcod=0'0)", 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.587550+0000", 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.915535826999999, 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001017783, 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587550+0000", 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587550+0000", 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587550+0000", 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587561+0000", 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1304999999999999e-05 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587573+0000", 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587585+0000", 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1815e-05 2024-09-13T07:12:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587637+0000", 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1788e-05 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.587663+0000", 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5948999999999998e-05 2024-09-13T07:12:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.588505+0000", 2024-09-13T07:12:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084248000000000005 2024-09-13T07:12:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.588568+0000", 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2748000000000003e-05 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4257.0:7 1.2 e17/14 1:5cea7035:::rbd_info:head v 17'2, mlcod=0'0)", 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.593006+0000", 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.9100799239999997, 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029468960000000001, 2024-09-13T07:12:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593006+0000", 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593006+0000", 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593006+0000", 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593012+0000", 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593023+0000", 2024-09-13T07:12:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1049e-05 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593038+0000", 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4598e-05 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593110+0000", 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2014999999999994e-05 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.593156+0000", 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6365000000000001e-05 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.595907+0000", 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002750541 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.595953+0000", 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6060000000000003e-05 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:32.163278+0000", 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.33980788899999997, 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045780329999999996, 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163278+0000", 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163278+0000", 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163276+0000", 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163287+0000", 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1096e-05 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163306+0000", 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8773000000000001e-05 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163315+0000", 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9900000000000003e-06 2024-09-13T07:12:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163383+0000", 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7677999999999998e-05 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.163392+0000", 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8549999999999997e-06 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.165931+0000", 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025387320000000001 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.166088+0000", 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015694099999999999 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.166319+0000", 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000231084 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167329+0000", 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001010087 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167787+0000", 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00045794899999999998 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167832+0000", 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167856+0000", 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4609000000000001e-05 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:11 3.1f e22/19 ondisk, result = 0)", 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:32.167221+0000", 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.33586450400000001, 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011153, 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167221+0000", 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167221+0000", 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167221+0000", 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167226+0000", 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-13T07:12:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167247+0000", 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1376e-05 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167272+0000", 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4379000000000002e-05 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167313+0000", 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0917999999999997e-05 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167322+0000", 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.167333+0000", 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1226999999999999e-05 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:32.177788+0000", 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.32529757300000001, 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032798400000000002, 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.177788+0000", 2024-09-13T07:12:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.177788+0000", 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.177786+0000", 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.177800+0000", 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3934000000000001e-05 2024-09-13T07:12:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.177820+0000", 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0176000000000001e-05 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.177836+0000", 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5690999999999999e-05 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.177901+0000", 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4839999999999996e-05 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178017+0000", 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116051 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178116+0000", 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9708000000000001e-05 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T07:12:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:32.178249+0000", 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.32483697, 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039216799999999998, 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 15 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178249+0000", 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178249+0000", 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178248+0000", 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178255+0000", 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4329999999999998e-06 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178271+0000", 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5464999999999998e-05 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178278+0000", 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.503e-06 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178315+0000", 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6863000000000003e-05 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178369+0000", 2024-09-13T07:12:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3899e-05 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:32.178641+0000", 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027201999999999999 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.531 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:32.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:32.564 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:32.565+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:32.664 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:32.665+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:32.679 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:32.679 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:32.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:32.766 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:32.766+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:32.835 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:32.835 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:32.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:32.866 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:32.867+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:32.952 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:32.952 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:32.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:32.952 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:32.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4257.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:12:32.952 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.368093622, 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01398422, 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4257", 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/806497074", 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:32.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584054+0000", 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584061+0000", 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1929999999999998e-06 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584069+0000", 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8240000000000006e-06 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584077+0000", 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584133+0000", 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5896000000000002e-05 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584145+0000", 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2293000000000001e-05 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593792+0000", 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096463009999999995 2024-09-13T07:12:32.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593914+0000", 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121994 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.594023+0000", 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109152 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597204+0000", 2024-09-13T07:12:32.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003181021 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597968+0000", 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076444399999999997 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597998+0000", 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0051000000000001e-05 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.598040+0000", 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1548999999999998e-05 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.882216+0000", 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.069933529, 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034400770000000002, 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882216+0000", 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882216+0000", 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882214+0000", 2024-09-13T07:12:32.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882227+0000", 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3443e-05 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882255+0000", 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7589999999999998e-05 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882267+0000", 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2323e-05 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882335+0000", 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7896000000000002e-05 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882356+0000", 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1471000000000002e-05 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.885611+0000", 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032548239999999999 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.885656+0000", 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.443e-05 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.04196172, 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044056909999999998, 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910194+0000", 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1439999999999997e-06 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910205+0000", 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1198e-05 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910210+0000", 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9590000000000003e-06 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910267+0000", 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7473999999999999e-05 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910291+0000", 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3456999999999999e-05 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914549+0000", 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042583059999999999 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914593+0000", 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3954e-05 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.041557331, 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040318009999999998, 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910591+0000", 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910597+0000", 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8200000000000002e-06 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910608+0000", 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1592e-05 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910613+0000", 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913555+0000", 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029417639999999999 2024-09-13T07:12:32.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913596+0000", 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0697e-05 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914598+0000", 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001001907 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914624+0000", 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5865000000000001e-05 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.041077809000001, 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039444399999999996, 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911070+0000", 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911098+0000", 2024-09-13T07:12:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7339999999999999e-05 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911112+0000", 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.385e-05 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911117+0000", 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9030000000000003e-06 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913900+0000", 2024-09-13T07:12:32.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002783761 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913923+0000", 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2313000000000001e-05 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914975+0000", 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010525319999999999 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.915016+0000", 2024-09-13T07:12:32.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0593999999999998e-05 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:111 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'67)", 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.091479+0000", 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.860670377, 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003085764, 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091479+0000", 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091479+0000", 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091478+0000", 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091482+0000", 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8809999999999998e-06 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091489+0000", 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091492+0000", 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.061e-06 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091523+0000", 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0583000000000002e-05 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091540+0000", 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7e-05 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094544+0000", 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030043169999999998 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094565+0000", 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0554999999999999e-05 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'67)", 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.860237335000001, 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054341140000000003, 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091911+0000", 2024-09-13T07:12:32.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091969+0000", 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7450000000000001e-05 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091976+0000", 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091979+0000", 2024-09-13T07:12:32.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2399999999999999e-06 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092008+0000", 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9040999999999999e-05 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092022+0000", 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3229e-05 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097327+0000", 2024-09-13T07:12:32.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053053470000000002 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097346+0000", 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9045000000000001e-05 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:113 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'67)", 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:32.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.859591175, 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048006739999999996, 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092557+0000", 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092562+0000", 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092572+0000", 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8530000000000004e-06 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092576+0000", 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4229999999999998e-06 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092608+0000", 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1735e-05 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092621+0000", 2024-09-13T07:12:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3268000000000001e-05 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097347+0000", 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047255589999999998 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097359+0000", 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1678999999999999e-05 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:114 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'76, mlcod=16'71)", 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.858951973, 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041827970000000003, 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093201+0000", 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8770000000000001e-06 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093211+0000", 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093216+0000", 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093240+0000", 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4624e-05 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093252+0000", 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1391e-05 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097360+0000", 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041079139999999998 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097380+0000", 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0392999999999999e-05 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:115 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'71)", 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.093565+0000", 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.858584142, 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003831611, 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093565+0000", 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093565+0000", 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093564+0000", 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093602+0000", 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7392999999999999e-05 2024-09-13T07:12:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093613+0000", 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1304000000000001e-05 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093618+0000", 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.622e-06 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093646+0000", 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8690000000000001e-05 2024-09-13T07:12:32.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093658+0000", 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1428e-05 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097381+0000", 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037229530000000002 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097397+0000", 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.588e-05 2024-09-13T07:12:32.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:116 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'78, mlcod=16'71)", 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.85814489, 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003406592, 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:32.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094009+0000", 2024-09-13T07:12:32.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094016+0000", 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2919999999999998e-06 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094020+0000", 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6200000000000001e-06 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094043+0000", 2024-09-13T07:12:32.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3166000000000001e-05 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094055+0000", 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2356e-05 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097397+0000", 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033420960000000001 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097411+0000", 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.362e-05 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:117 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'79, mlcod=16'71)", 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857778979999999, 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058774140000000001, 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094374+0000", 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0720000000000001e-06 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094385+0000", 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.146e-05 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094390+0000", 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-06 2024-09-13T07:12:33.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094417+0000", 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7350000000000001e-05 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094427+0000", 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0257e-05 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.100222+0000", 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057946849999999999 2024-09-13T07:12:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.100248+0000", 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5565e-05 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:131 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.442112903, 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028894860000000001, 2024-09-13T07:12:33.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510287+0000", 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025087299999999998 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510296+0000", 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3670000000000003e-06 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510300+0000", 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9650000000000002e-06 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510326+0000", 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6057000000000001e-05 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510339+0000", 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2848e-05 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.512892+0000", 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025529709999999998 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.512926+0000", 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3951000000000001e-05 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:132 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.441140679, 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029677789999999998, 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511012+0000", 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6289999999999999e-06 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511018+0000", 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511021+0000", 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0489999999999999e-06 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511061+0000", 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9978999999999998e-05 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511081+0000", 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9854e-05 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.513950+0000", 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028695109999999999 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.513976+0000", 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5795e-05 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:139 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'93)", 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.435860676000001, 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035917100000000001, 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516293+0000", 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516302+0000", 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2990000000000002e-06 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516307+0000", 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5399999999999997e-06 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516333+0000", 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5989000000000001e-05 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516344+0000", 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1005000000000001e-05 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519852+0000", 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035081660000000001 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519880+0000", 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8095000000000001e-05 2024-09-13T07:12:33.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:140 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'93)", 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.516838+0000", 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.435311315, 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034447950000000001, 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516838+0000", 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516838+0000", 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516837+0000", 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516915+0000", 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7947000000000004e-05 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516922+0000", 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3419999999999996e-06 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516925+0000", 2024-09-13T07:12:33.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6320000000000001e-06 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516982+0000", 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6737000000000002e-05 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517005+0000", 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2566e-05 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.520254+0000", 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032491949999999999 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.520283+0000", 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8887999999999999e-05 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:141 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'95)", 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.434193523999999, 2024-09-13T07:12:33.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052230660000000002, 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517955+0000", 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517961+0000", 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6149999999999996e-06 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517968+0000", 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.875e-06 2024-09-13T07:12:33.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517973+0000", 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4150000000000001e-06 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519906+0000", 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001932879 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519919+0000", 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2877999999999999e-05 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.523151+0000", 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032328880000000002 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.523179+0000", 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7209999999999999e-05 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:143 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-13T07:12:33.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.427278701000001, 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033785579999999998, 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524957+0000", 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7517999999999998e-05 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524965+0000", 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2649999999999999e-06 2024-09-13T07:12:33.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524968+0000", 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7230000000000002e-06 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525035+0000", 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6841000000000006e-05 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525061+0000", 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5622999999999998e-05 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.528214+0000", 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003152593 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.528249+0000", 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5475999999999997e-05 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:144 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.426682792999999, 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052055089999999997, 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525530+0000", 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4370999999999997e-05 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525539+0000", 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0510000000000002e-06 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525543+0000", 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9310000000000001e-06 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525575+0000", 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1359000000000002e-05 2024-09-13T07:12:33.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525586+0000", 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530640+0000", 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050537189999999999 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530672+0000", 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1887999999999997e-05 2024-09-13T07:12:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:145 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.426195605, 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047501109999999996, 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525953+0000", 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526013+0000", 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0170000000000002e-05 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526021+0000", 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5390000000000002e-06 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526024+0000", 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2049999999999998e-06 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526057+0000", 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.345e-05 2024-09-13T07:12:33.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526070+0000", 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2315e-05 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530675+0000", 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004605217 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530704+0000", 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.866e-05 2024-09-13T07:12:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:33.039 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:33.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:33.041 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:33.041+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:33.141 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:33.141+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:33.190 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:33.190 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:33.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:33.242 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:33.242+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:33.342 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:33.343+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:33.347 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:33.347 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:33.347 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:33.347 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:33.347 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:33.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:33.444 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:33.445+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:33.504 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:33.504 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:33.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:33.504 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [] 2024-09-13T07:12:33.505 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:33.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:33.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:12:33.555 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:33.556+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:33.646 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:33.646+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:33.669 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:33.669 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:33.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:33.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:33.685 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:33.702 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:33.702 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:33.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:33.720 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:33.736 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:33.736 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:33.746 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:33.747+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:33.753 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:33.753 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:33.770 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:33.770 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:33.788 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:33.788 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:33.788 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:33.798 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:33.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:12:33.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:12:33.857 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:33.858+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:1 write 0xdeb203a thru 0xdeb9f04 (0x7ecb bytes) 2024-09-13T07:12:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:2 read 0x34eb6c2 thru 0x34f903d (0xd97c bytes) 2024-09-13T07:12:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:33.931 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:33.933 INFO:teuthology.orchestra.run.smithi031.stdout:4 writesame 0xded8a2a thru 0xdee6dae data_size 0x37(0xe385 bytes) 2024-09-13T07:12:33.934 INFO:teuthology.orchestra.run.smithi031.stdout:5 read 0x413c05b thru 0x414ab93 (0xeb39 bytes) 2024-09-13T07:12:33.935 INFO:teuthology.orchestra.run.smithi031.stdout:6 write 0x76c8279 thru 0x76d6ab1 (0xe839 bytes) 2024-09-13T07:12:33.940 INFO:teuthology.orchestra.run.smithi031.stdout:7 write 0xdc1ec4d thru 0xdc1f717 (0xacb bytes) 2024-09-13T07:12:33.948 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:33.949+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:33.949 INFO:teuthology.orchestra.run.smithi031.stdout:8 write 0xec2bdb1 thru 0xec32366 (0x65b6 bytes) 2024-09-13T07:12:33.953 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xcd46606 2024-09-13T07:12:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:9 trunc from 0xec32367 to 0xcd46606 2024-09-13T07:12:33.976 INFO:teuthology.orchestra.run.smithi031.stdout:10 trunc from 0xcd46606 to 0x89d9eaf 2024-09-13T07:12:33.983 INFO:teuthology.orchestra.run.smithi031.stdout:12 read 0x25969e7 thru 0x259c30c (0x5926 bytes) 2024-09-13T07:12:33.983 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:33.994 INFO:teuthology.orchestra.run.smithi031.stdout:14 writesame 0xae0e028 thru 0xae1865c data_size 0x3(0xa635 bytes) 2024-09-13T07:12:34.003 INFO:teuthology.orchestra.run.smithi031.stdout:15 writesame 0xc12a05d thru 0xc12c21f data_size 0x2b(0x21c3 bytes) 2024-09-13T07:12:34.003 INFO:teuthology.orchestra.run.smithi031.stdout:16 write 0xa98add6 thru 0xa98d747 (0x2972 bytes) 2024-09-13T07:12:34.006 INFO:teuthology.orchestra.run.smithi031.stdout:17 write 0x5388de1 thru 0x538a402 (0x1622 bytes) 2024-09-13T07:12:34.010 INFO:teuthology.orchestra.run.smithi031.stdout:18 writesame 0x896c1e5 thru 0x8973c21 data_size 0x3d(0x7a3d bytes) 2024-09-13T07:12:34.013 INFO:teuthology.orchestra.run.smithi031.stdout:19 write 0x19397da thru 0x19445f9 (0xae20 bytes) 2024-09-13T07:12:34.017 INFO:teuthology.orchestra.run.smithi031.stdout:20 read 0x6045189 thru 0x6048fe4 (0x3e5c bytes) 2024-09-13T07:12:34.017 INFO:teuthology.orchestra.run.smithi031.stdout:22 punch from 0x3efd5be to 0x3f09121, (0xbb63 bytes) 2024-09-13T07:12:34.017 INFO:teuthology.orchestra.run.smithi031.stdout:24 write 0x3f742e2 thru 0x3f8386e (0xf58d bytes) 2024-09-13T07:12:34.021 INFO:teuthology.orchestra.run.smithi031.stdout:25 read 0x8fc3616 thru 0x8fc7fa8 (0x4993 bytes) 2024-09-13T07:12:34.031 INFO:teuthology.orchestra.run.smithi031.stdout:27 write 0xed32166 thru 0xed37c3b (0x5ad6 bytes) 2024-09-13T07:12:34.046 INFO:teuthology.orchestra.run.smithi031.stdout:28 trunc from 0xed37c3c to 0x3756cc4 2024-09-13T07:12:34.048 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.048 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.049 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:34.050+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:34.059 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:34.059 INFO:teuthology.orchestra.run.smithi031.stdout:30 trunc from 0x3756cc4 to 0x513eedf 2024-09-13T07:12:34.065 INFO:teuthology.orchestra.run.smithi031.stdout:32 write 0x38c0038 thru 0x38c9728 (0x96f1 bytes) 2024-09-13T07:12:34.078 INFO:teuthology.orchestra.run.smithi031.stdout:33 write 0x893d753 thru 0x893f84e (0x20fc bytes) 2024-09-13T07:12:34.081 INFO:teuthology.orchestra.run.smithi031.stdout:36 read 0x3c8460b thru 0x3c89103 (0x4af9 bytes) 2024-09-13T07:12:34.081 INFO:teuthology.orchestra.run.smithi031.stdout:37 read 0x23bfa05 thru 0x23cf6f5 (0xfcf1 bytes) 2024-09-13T07:12:34.081 INFO:teuthology.orchestra.run.smithi031.stdout:38 write 0x3a950bf thru 0x3a9bf47 (0x6e89 bytes) 2024-09-13T07:12:34.082 INFO:teuthology.orchestra.run.smithi031.stdout:39 write 0x7731337 thru 0x773b7d8 (0xa4a2 bytes) 2024-09-13T07:12:34.088 INFO:teuthology.orchestra.run.smithi031.stdout:40 trunc from 0x893f84f to 0xbcc7efd 2024-09-13T07:12:34.093 INFO:teuthology.orchestra.run.smithi031.stdout:41 write 0x7ce0d6e thru 0x7cec23f (0xb4d2 bytes) 2024-09-13T07:12:34.096 INFO:teuthology.orchestra.run.smithi031.stdout:42 write 0x9f2c5ed thru 0x9f33afb (0x750f bytes) 2024-09-13T07:12:34.099 INFO:teuthology.orchestra.run.smithi031.stdout:44 write 0x8b2a7bc thru 0x8b35d6c (0xb5b1 bytes) 2024-09-13T07:12:34.101 INFO:teuthology.orchestra.run.smithi031.stdout:45 read 0x69fc08b thru 0x6a0000f (0x3f85 bytes) 2024-09-13T07:12:34.101 INFO:teuthology.orchestra.run.smithi031.stdout:46 writesame 0xb6fffa6 thru 0xb70b618 data_size 0x1(0xb673 bytes) 2024-09-13T07:12:34.125 INFO:teuthology.orchestra.run.smithi031.stdout:48 write 0x642a3e3 thru 0x6431320 (0x6f3e bytes) 2024-09-13T07:12:34.128 INFO:teuthology.orchestra.run.smithi031.stdout:49 write 0x7d72081 thru 0x7d77926 (0x58a6 bytes) 2024-09-13T07:12:34.130 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:34.130 INFO:teuthology.orchestra.run.smithi031.stdout:51 writesame 0x52a39d4 thru 0x52a8710 data_size 0x27(0x4d3d bytes) 2024-09-13T07:12:34.138 INFO:teuthology.orchestra.run.smithi031.stdout:53 write 0xc4b1937 thru 0xc4b98c5 (0x7f8f bytes) 2024-09-13T07:12:34.142 INFO:teuthology.orchestra.run.smithi031.stdout:54 punch from 0xb4809 to 0xb7271, (0x2a68 bytes) 2024-09-13T07:12:34.142 INFO:teuthology.orchestra.run.smithi031.stdout:55 read 0x9a70053 thru 0x9a72540 (0x24ee bytes) 2024-09-13T07:12:34.142 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xdf4fc14 2024-09-13T07:12:34.144 INFO:teuthology.orchestra.run.smithi031.stdout:60 trunc from 0xc4b98c6 to 0xdf4fc14 2024-09-13T07:12:34.149 INFO:teuthology.orchestra.run.smithi031.stdout:61 write 0x716ffcf thru 0x71715ef (0x1621 bytes) 2024-09-13T07:12:34.149 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:34.150+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:34.152 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:34.152 INFO:teuthology.orchestra.run.smithi031.stdout:63 punch from 0x9cb40e7 to 0x9cc19d6, (0xd8ef bytes) 2024-09-13T07:12:34.158 INFO:teuthology.orchestra.run.smithi031.stdout:65 write 0xee47ac7 thru 0xee56d2f (0xf269 bytes) 2024-09-13T07:12:34.161 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:34.161 INFO:teuthology.orchestra.run.smithi031.stdout:69 read 0x26d37b8 thru 0x26d635f (0x2ba8 bytes) 2024-09-13T07:12:34.162 INFO:teuthology.orchestra.run.smithi031.stdout:71 clone 1 order 20 su 131072 sc 5 2024-09-13T07:12:34.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:12:34.205 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.205 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.213 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:34.255 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:34.256+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:34.352 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:34.353+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:34.356 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.356 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.365 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:34.452 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:34.453+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:34.466 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.467 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.475 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:34.499 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:12:34.500 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:12:34.500 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T07:12:34.500 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:12:34.500 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:12:34.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T07:12:34.554 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:34.555+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:34.576 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.577 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.585 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:34.606 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:12:34.607 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:34.656 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:34.657+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:34.687 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.687 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.696 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:34.756 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:34.757+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:34.798 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.798 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.807 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:34.891 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:34.892+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:34.949 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:34.949 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:34.958 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:34.958 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:34.959+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:35.060 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:35.061+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:35.100 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:35.100 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:35.109 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:12:35.161 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:35.162+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:12:35.219 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:35.219 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:35.229 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:12:35.263 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:35.263+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:35.363 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:35.364+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:35.373 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:35.373 INFO:teuthology.orchestra.run.smithi031.stdout:73 read 0x4c9a355 thru 0x4ca1d3b (0x79e7 bytes) 2024-09-13T07:12:35.373 INFO:teuthology.orchestra.run.smithi031.stdout:74 write 0x63238d6 thru 0x6326ea1 (0x35cc bytes) 2024-09-13T07:12:35.376 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:35.376 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:35.380 INFO:teuthology.orchestra.run.smithi031.stdout:75 read 0xd6aaa68 thru 0xd6abbf3 (0x118c bytes) 2024-09-13T07:12:35.380 INFO:teuthology.orchestra.run.smithi031.stdout:76 write 0xd0ed420 thru 0xd0fb07a (0xdc5b bytes) 2024-09-13T07:12:35.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:12:35.386 INFO:teuthology.orchestra.run.smithi031.stdout:77 read 0x1899068 thru 0x18a2588 (0x9521 bytes) 2024-09-13T07:12:35.388 INFO:teuthology.orchestra.run.smithi031.stdout:78 writesame 0x70b717c thru 0x70c16fc data_size 0x1d(0xa581 bytes) 2024-09-13T07:12:35.394 INFO:teuthology.orchestra.run.smithi031.stdout:80 read 0xdf0b045 thru 0xdf1439b (0x9357 bytes) 2024-09-13T07:12:35.394 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:35.394 INFO:teuthology.orchestra.run.smithi031.stdout:82 writesame 0x976266f thru 0x9766901 data_size 0x27(0x4293 bytes) 2024-09-13T07:12:35.398 INFO:teuthology.orchestra.run.smithi031.stdout:85 read 0xd486d6d thru 0xd48aa7a (0x3d0e bytes) 2024-09-13T07:12:35.398 INFO:teuthology.orchestra.run.smithi031.stdout:87 writesame 0x87b45e5 thru 0x87b6789 data_size 0x21(0x21a5 bytes) 2024-09-13T07:12:35.402 INFO:teuthology.orchestra.run.smithi031.stdout:88 read 0x7ab2197 thru 0x7ab6e14 (0x4c7e bytes) 2024-09-13T07:12:35.464 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:35.465+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:35.464 INFO:teuthology.orchestra.run.smithi031.stdout:89 write 0x177bd7e thru 0x17867e5 (0xaa68 bytes) 2024-09-13T07:12:35.471 INFO:teuthology.orchestra.run.smithi031.stdout:90 write 0xb59ccab thru 0xb5a32b9 (0x660f bytes) 2024-09-13T07:12:35.478 INFO:teuthology.orchestra.run.smithi031.stdout:91 write 0x65e4720 thru 0x65ecd29 (0x860a bytes) 2024-09-13T07:12:35.489 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:35.489 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:35.497 INFO:teuthology.orchestra.run.smithi031.stdout:94 trunc from 0xee56d30 to 0x1fe8a48 2024-09-13T07:12:35.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:35.527 INFO:teuthology.orchestra.run.smithi031.stdout:95 punch from 0x67a175 to 0x67f1ce, (0x5059 bytes) 2024-09-13T07:12:35.527 INFO:teuthology.orchestra.run.smithi031.stdout:96 read 0x14052bf thru 0x1414f37 (0xfc79 bytes) 2024-09-13T07:12:35.527 INFO:teuthology.orchestra.run.smithi031.stdout:97 punch from 0xe1bd1d to 0xe24ff0, (0x92d3 bytes) 2024-09-13T07:12:35.536 INFO:teuthology.orchestra.run.smithi031.stdout:98 write 0x536d21f thru 0x537709b (0x9e7d bytes) 2024-09-13T07:12:35.539 INFO:teuthology.orchestra.run.smithi031.stdout:100 read 0x49cb94b thru 0x49d76b0 (0xbd66 bytes) 2024-09-13T07:12:35.539 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:35.539 INFO:teuthology.orchestra.run.smithi031.stdout:102 read 0x43d1234 thru 0x43dbcd3 (0xaaa0 bytes) 2024-09-13T07:12:35.544 INFO:teuthology.orchestra.run.smithi031.stdout:106 trunc from 0x537709c to 0xcc0382 2024-09-13T07:12:35.565 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:35.566+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:35.570 INFO:teuthology.orchestra.run.smithi031.stdout:108 read 0xc36a03 thru 0xc3be66 (0x5464 bytes) 2024-09-13T07:12:35.578 INFO:teuthology.orchestra.run.smithi031.stdout:110 write 0x3101324 thru 0x310695f (0x563c bytes) 2024-09-13T07:12:35.586 INFO:teuthology.orchestra.run.smithi031.stdout:112 write 0x3124555 thru 0x312cdad (0x8859 bytes) 2024-09-13T07:12:35.589 INFO:teuthology.orchestra.run.smithi031.stdout:113 clone 2 order 19 su 262144 sc 8 2024-09-13T07:12:35.614 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:35.614 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:35.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:35.665 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:35.666+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:35.766 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:35.767+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:35.770 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:35.770 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:35.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:35.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:12:35.867 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:35.868+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:35.927 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:35.927 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:35.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:35.942 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:35.960 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:35.960 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:35.967 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:35.968+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:35.976 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:35.992 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:36.009 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:36.026 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:36.027 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:36.043 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:36.044 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:36.044 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:36.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:12:36.060 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:12:36.068 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:36.069+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:36.169 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:36.170+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:36.169 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:36.169 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:36.169 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:36.170 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:36.170 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:36.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:36.226 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone1 2024-09-13T07:12:36.270 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:36.271+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:36.285 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4257.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.754746071, 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01398422, 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4257", 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/806497074", 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584054+0000", 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584061+0000", 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1929999999999998e-06 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584069+0000", 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8240000000000006e-06 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584077+0000", 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584133+0000", 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5896000000000002e-05 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584145+0000", 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2293000000000001e-05 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593792+0000", 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096463009999999995 2024-09-13T07:12:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593914+0000", 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121994 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.594023+0000", 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109152 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597204+0000", 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003181021 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597968+0000", 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076444399999999997 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597998+0000", 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0051000000000001e-05 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.598040+0000", 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1548999999999998e-05 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.882216+0000", 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.456585978, 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034400770000000002, 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882216+0000", 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882216+0000", 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882214+0000", 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882227+0000", 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3443e-05 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882255+0000", 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7589999999999998e-05 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882267+0000", 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2323e-05 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882335+0000", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7896000000000002e-05 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.882356+0000", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1471000000000002e-05 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.885611+0000", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032548239999999999 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.885656+0000", 2024-09-13T07:12:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.443e-05 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.428614168999999, 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044056909999999998, 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910194+0000", 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1439999999999997e-06 2024-09-13T07:12:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910205+0000", 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1198e-05 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910210+0000", 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9590000000000003e-06 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910267+0000", 2024-09-13T07:12:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7473999999999999e-05 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910291+0000", 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3456999999999999e-05 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914549+0000", 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042583059999999999 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914593+0000", 2024-09-13T07:12:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3954e-05 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.42820978, 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040318009999999998, 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910591+0000", 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910597+0000", 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8200000000000002e-06 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910608+0000", 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1592e-05 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910613+0000", 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-13T07:12:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913555+0000", 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029417639999999999 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913596+0000", 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0697e-05 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914598+0000", 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001001907 2024-09-13T07:12:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914624+0000", 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5865000000000001e-05 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.427730258, 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039444399999999996, 2024-09-13T07:12:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911070+0000", 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911098+0000", 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7339999999999999e-05 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911112+0000", 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.385e-05 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911117+0000", 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9030000000000003e-06 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913900+0000", 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002783761 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913923+0000", 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2313000000000001e-05 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914975+0000", 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010525319999999999 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.915016+0000", 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0593999999999998e-05 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:111 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'67)", 2024-09-13T07:12:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.091479+0000", 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.247322826, 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003085764, 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091479+0000", 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091479+0000", 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091478+0000", 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091482+0000", 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8809999999999998e-06 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091489+0000", 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091492+0000", 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.061e-06 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091523+0000", 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0583000000000002e-05 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091540+0000", 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7e-05 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094544+0000", 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030043169999999998 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094565+0000", 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0554999999999999e-05 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'67)", 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.246889784, 2024-09-13T07:12:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054341140000000003, 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091911+0000", 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091969+0000", 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7450000000000001e-05 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091976+0000", 2024-09-13T07:12:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091979+0000", 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2399999999999999e-06 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092008+0000", 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9040999999999999e-05 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092022+0000", 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3229e-05 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097327+0000", 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053053470000000002 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097346+0000", 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9045000000000001e-05 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:113 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'67)", 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.246243624, 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048006739999999996, 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092557+0000", 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092562+0000", 2024-09-13T07:12:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092572+0000", 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8530000000000004e-06 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092576+0000", 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4229999999999998e-06 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092608+0000", 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1735e-05 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092621+0000", 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3268000000000001e-05 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097347+0000", 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047255589999999998 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097359+0000", 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1678999999999999e-05 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:114 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'76, mlcod=16'71)", 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.245604422, 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041827970000000003, 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093201+0000", 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8770000000000001e-06 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093211+0000", 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093216+0000", 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093240+0000", 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4624e-05 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093252+0000", 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1391e-05 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097360+0000", 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041079139999999998 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097380+0000", 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0392999999999999e-05 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:115 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'71)", 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.093565+0000", 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.245236590999999, 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003831611, 2024-09-13T07:12:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093565+0000", 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093565+0000", 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093564+0000", 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093602+0000", 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7392999999999999e-05 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093613+0000", 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1304000000000001e-05 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093618+0000", 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.622e-06 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093646+0000", 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8690000000000001e-05 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093658+0000", 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1428e-05 2024-09-13T07:12:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097381+0000", 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037229530000000002 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097397+0000", 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.588e-05 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:116 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'78, mlcod=16'71)", 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.244797339, 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003406592, 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094004+0000", 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094009+0000", 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094016+0000", 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2919999999999998e-06 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094020+0000", 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6200000000000001e-06 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094043+0000", 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3166000000000001e-05 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094055+0000", 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2356e-05 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097397+0000", 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033420960000000001 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097411+0000", 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.362e-05 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:117 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'79, mlcod=16'71)", 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.244431429, 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058774140000000001, 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094374+0000", 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0720000000000001e-06 2024-09-13T07:12:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094385+0000", 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.146e-05 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094390+0000", 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-06 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094417+0000", 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7350000000000001e-05 2024-09-13T07:12:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094427+0000", 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0257e-05 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.100222+0000", 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057946849999999999 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.100248+0000", 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5565e-05 2024-09-13T07:12:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:131 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.828765352, 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028894860000000001, 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510036+0000", 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510287+0000", 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025087299999999998 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510296+0000", 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3670000000000003e-06 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510300+0000", 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9650000000000002e-06 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510326+0000", 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6057000000000001e-05 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.510339+0000", 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2848e-05 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.512892+0000", 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025529709999999998 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.512926+0000", 2024-09-13T07:12:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3951000000000001e-05 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:132 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.827793128, 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029677789999999998, 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511008+0000", 2024-09-13T07:12:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511012+0000", 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6289999999999999e-06 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511018+0000", 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511021+0000", 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0489999999999999e-06 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511061+0000", 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9978999999999998e-05 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.511081+0000", 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9854e-05 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.513950+0000", 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028695109999999999 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.513976+0000", 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5795e-05 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:139 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'93)", 2024-09-13T07:12:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.822513125, 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035917100000000001, 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516288+0000", 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516293+0000", 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516302+0000", 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2990000000000002e-06 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516307+0000", 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5399999999999997e-06 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516333+0000", 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5989000000000001e-05 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516344+0000", 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1005000000000001e-05 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519852+0000", 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035081660000000001 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519880+0000", 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8095000000000001e-05 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:140 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'93)", 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.516838+0000", 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.821963763999999, 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034447950000000001, 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516838+0000", 2024-09-13T07:12:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516838+0000", 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516837+0000", 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516915+0000", 2024-09-13T07:12:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7947000000000004e-05 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516922+0000", 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3419999999999996e-06 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516925+0000", 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6320000000000001e-06 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.516982+0000", 2024-09-13T07:12:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6737000000000002e-05 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517005+0000", 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2566e-05 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.520254+0000", 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032491949999999999 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.520283+0000", 2024-09-13T07:12:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8887999999999999e-05 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:141 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'95)", 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.820845973000001, 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052230660000000002, 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517955+0000", 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517961+0000", 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6149999999999996e-06 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517968+0000", 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.875e-06 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517973+0000", 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4150000000000001e-06 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519906+0000", 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001932879 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519919+0000", 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2877999999999999e-05 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.523151+0000", 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032328880000000002 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.523179+0000", 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7209999999999999e-05 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:143 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.81393115, 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033785579999999998, 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524870+0000", 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524957+0000", 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7517999999999998e-05 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524965+0000", 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2649999999999999e-06 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.524968+0000", 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7230000000000002e-06 2024-09-13T07:12:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525035+0000", 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6841000000000006e-05 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525061+0000", 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5622999999999998e-05 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.528214+0000", 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003152593 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.528249+0000", 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5475999999999997e-05 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:144 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.813335242000001, 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052055089999999997, 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525530+0000", 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4370999999999997e-05 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525539+0000", 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0510000000000002e-06 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525543+0000", 2024-09-13T07:12:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9310000000000001e-06 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525575+0000", 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1359000000000002e-05 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525586+0000", 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530640+0000", 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050537189999999999 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530672+0000", 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1887999999999997e-05 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:145 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.812848054, 2024-09-13T07:12:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047501109999999996, 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525953+0000", 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526013+0000", 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0170000000000002e-05 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526021+0000", 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5390000000000002e-06 2024-09-13T07:12:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526024+0000", 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2049999999999998e-06 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526057+0000", 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.345e-05 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526070+0000", 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2315e-05 2024-09-13T07:12:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530675+0000", 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004605217 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530704+0000", 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.866e-05 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.430 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:36.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:36.432 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:36.432+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:36.531 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:36.532+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:36.547 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:36.547 INFO:teuthology.orchestra.run.smithi031.stdout:116 read 0x11ba90e thru 0x11c6620 (0xbd13 bytes) 2024-09-13T07:12:36.547 INFO:teuthology.orchestra.run.smithi031.stdout:117 punch from 0x1b999f5 to 0x1ba916c, (0xf777 bytes) 2024-09-13T07:12:36.558 INFO:teuthology.orchestra.run.smithi031.stdout:119 write 0x530d486 thru 0x531202d (0x4ba8 bytes) 2024-09-13T07:12:36.575 INFO:teuthology.orchestra.run.smithi031.stdout:120 write 0xcac83be thru 0xcacd5f7 (0x523a bytes) 2024-09-13T07:12:36.578 INFO:teuthology.orchestra.run.smithi031.stdout:121 read 0x6dd6f93 thru 0x6de242e (0xb49c bytes) 2024-09-13T07:12:36.578 INFO:teuthology.orchestra.run.smithi031.stdout:123 write 0x182706d thru 0x182edf1 (0x7d85 bytes) 2024-09-13T07:12:36.580 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:36.581 INFO:teuthology.orchestra.run.smithi031.stdout:125 punch from 0x83e4620 to 0x83f19d6, (0xd3b6 bytes) 2024-09-13T07:12:36.586 INFO:teuthology.orchestra.run.smithi031.stdout:126 writesame 0xd82a061 thru 0xd830d1c data_size 0x2(0x6cbc bytes) 2024-09-13T07:12:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:127 read 0x555e704 thru 0x556309f (0x499c bytes) 2024-09-13T07:12:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:129 write 0xae9b1a7 thru 0xaeaaf21 (0xfd7b bytes) 2024-09-13T07:12:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:132 read 0xa43ceec thru 0xa448d7e (0xbe93 bytes) 2024-09-13T07:12:36.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:36.601 INFO:teuthology.orchestra.run.smithi031.stdout:133 trunc from 0xd830d1d to 0x33cd46b 2024-09-13T07:12:36.611 INFO:teuthology.orchestra.run.smithi031.stdout:134 read 0x3057f5f thru 0x305b2d8 (0x337a bytes) 2024-09-13T07:12:36.630 INFO:teuthology.orchestra.run.smithi031.stdout:135 write 0xe991218 thru 0xe992f34 (0x1d1d bytes) 2024-09-13T07:12:36.632 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:36.633+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:36.633 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:36.633 INFO:teuthology.orchestra.run.smithi031.stdout:139 write 0x852a590 thru 0x8534a70 (0xa4e1 bytes) 2024-09-13T07:12:36.636 INFO:teuthology.orchestra.run.smithi031.stdout:140 writesame 0xa3d134e thru 0xa3d8291 data_size 0x2(0x6f44 bytes) 2024-09-13T07:12:36.644 INFO:teuthology.orchestra.run.smithi031.stdout:142 read 0xa43f56b thru 0xa4413bb (0x1e51 bytes) 2024-09-13T07:12:36.644 INFO:teuthology.orchestra.run.smithi031.stdout:144 writesame 0xeb1e23 thru 0xebeeeb data_size 0xb(0xd0c9 bytes) 2024-09-13T07:12:36.649 INFO:teuthology.orchestra.run.smithi031.stdout:145 read 0xfce0c thru 0x1061c2 (0x93b7 bytes) 2024-09-13T07:12:36.649 INFO:teuthology.orchestra.run.smithi031.stdout:146 read 0x902b78f thru 0x902e4c5 (0x2d37 bytes) 2024-09-13T07:12:36.649 INFO:teuthology.orchestra.run.smithi031.stdout:147 write 0xb62ecf1 thru 0xb636e19 (0x8129 bytes) 2024-09-13T07:12:36.653 INFO:teuthology.orchestra.run.smithi031.stdout:148 write 0x75f4703 thru 0x76015de (0xcedc bytes) 2024-09-13T07:12:36.660 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:36.660 INFO:teuthology.orchestra.run.smithi031.stdout:151 writesame 0x768fccc thru 0x7696083 data_size 0x4(0x63b8 bytes) 2024-09-13T07:12:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:152 read 0x2591f thru 0x2d6de (0x7dc0 bytes) 2024-09-13T07:12:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:154 read 0x2aa70fb thru 0x2ab589c (0xe7a2 bytes) 2024-09-13T07:12:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:155 write 0xaf1bd28 thru 0xaf1e76b (0x2a44 bytes) 2024-09-13T07:12:36.670 INFO:teuthology.orchestra.run.smithi031.stdout:156 write 0x55e1374 thru 0x55e486c (0x34f9 bytes) 2024-09-13T07:12:36.674 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:36.674 INFO:teuthology.orchestra.run.smithi031.stdout:160 clone 3 order 20 su 262144 sc 15 2024-09-13T07:12:36.715 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:36.715 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:36.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:36.764 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:36.765+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:36.834 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:36.835+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:36.869 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:571 3.0 e24/19 3:04479396:::rbd_id.image_client.0-clone2:head v 24'4, mlcod=22'2)", 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:36.203625+0000", 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66763371800000004, 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078267300000000001, 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203625+0000", 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203625+0000", 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203625+0000", 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203631+0000", 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3080000000000004e-06 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203648+0000", 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7394999999999999e-05 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203658+0000", 2024-09-13T07:12:36.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8819999999999996e-06 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203705+0000", 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6640000000000001e-05 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.203720+0000", 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5147000000000001e-05 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.204379+0000", 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065925200000000004 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.204407+0000", 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8313000000000001e-05 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:789 3.9 e24/19 3:9280c38f:::rbd_data.1173f77ecad8.000000000000010c:head v 24'3, mlcod=23'1)", 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:36.637141+0000", 2024-09-13T07:12:36.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.23411758299999999, 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087117599999999998, 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637141+0000", 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637141+0000", 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637140+0000", 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637241+0000", 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010092199999999999 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637257+0000", 2024-09-13T07:12:36.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6523000000000001e-05 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637264+0000", 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637308+0000", 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4521000000000003e-05 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637325+0000", 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7221999999999999e-05 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637968+0000", 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064206599999999995 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.638012+0000", 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4589000000000003e-05 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:802 3.9 e24/19 3:933222f0:::rbd_data.1173f77ecad8.00000000000000e8:head v 24'4, mlcod=23'2)", 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:36.657279+0000", 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.21397981999999999, 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063402700000000001, 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:36.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657279+0000", 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657279+0000", 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657277+0000", 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:36.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657289+0000", 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1681e-05 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657304+0000", 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5036e-05 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657312+0000", 2024-09-13T07:12:36.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6809999999999998e-06 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657351+0000", 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9004000000000001e-05 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657370+0000", 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8953999999999998e-05 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657861+0000", 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049171299999999996 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.657913+0000", 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1329999999999998e-05 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:36.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:36.884 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:36.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:36.969 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:36.970+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:37.031 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:37.031 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:37.032 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:37.032 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:37.036 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:37.037+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:37.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:37.137 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:37.138+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:37.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:12:37.189 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:37.189 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:37.190 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:37.190 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:37.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:37.236 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone2 from 0xe992f35 (overlap 0x312cdae) to 0x0 2024-09-13T07:12:37.238 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:37.239+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:37.339 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:37.339+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:913 3.1e 3:7c3d5185:::rbd_data.1173f77ecad8.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.245992+0000", 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.12669365699999999, 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052291789999999996, 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 913 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245992+0000", 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245992+0000", 2024-09-13T07:12:37.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245991+0000", 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245997+0000", 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246010+0000", 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2422000000000001e-05 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246018+0000", 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0069999999999997e-06 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246075+0000", 2024-09-13T07:12:37.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-05 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246158+0000", 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2939e-05 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246372+0000", 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000214198 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248033+0000", 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016612720000000001 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251112+0000", 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030792559999999998 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251172+0000", 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9700000000000001e-05 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251221+0000", 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9444000000000003e-05 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:923 3.7 3:e0d0441f:::rbd_data.1173f77ecad8.0000000000000010:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.120900411, 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054522629999999997, 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 923 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251784+0000", 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251790+0000", 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8300000000000001e-06 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251802+0000", 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1935e-05 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251807+0000", 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251857+0000", 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9515000000000003e-05 2024-09-13T07:12:37.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.252009+0000", 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015179700000000001 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.257238+0000", 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052290870000000003 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1018 3.2 3:445402e0:::rbd_data.1173f77ecad8.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271055+0000", 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.101630759, 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050128129999999996, 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1018 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271055+0000", 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271055+0000", 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271054+0000", 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271068+0000", 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3923999999999999e-05 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271083+0000", 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4755e-05 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271091+0000", 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3510000000000002e-06 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271145+0000", 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4206000000000002e-05 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271224+0000", 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8722999999999998e-05 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271430+0000", 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206565 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.272720+0000", 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012896850000000001 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.275974+0000", 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032543519999999999 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276027+0000", 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379000000000001e-05 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276068+0000", 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1114e-05 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1020 3.8 3:11bc453b:::rbd_data.1173f77ecad8.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271120+0000", 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.101566237, 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051601049999999999, 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1020 2024-09-13T07:12:37.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271120+0000", 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271120+0000", 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271119+0000", 2024-09-13T07:12:37.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271123+0000", 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3340000000000003e-06 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271133+0000", 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2280000000000004e-06 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271139+0000", 2024-09-13T07:12:37.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.02e-06 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271184+0000", 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5031000000000003e-05 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271262+0000", 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8380999999999994e-05 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271455+0000", 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019298199999999999 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276096+0000", 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046414300000000002 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276191+0000", 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4306000000000001e-05 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276241+0000", 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0614e-05 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276280+0000", 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8387999999999999e-05 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1021 3.b 3:d4309e9e:::rbd_data.1173f77ecad8.000000000000002d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271673+0000", 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.101013389, 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053003850000000003, 2024-09-13T07:12:37.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1021 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271673+0000", 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271673+0000", 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271672+0000", 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271681+0000", 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5909999999999996e-06 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271689+0000", 2024-09-13T07:12:37.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271701+0000", 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1857e-05 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271767+0000", 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6196000000000001e-05 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271912+0000", 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014517100000000001 2024-09-13T07:12:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276973+0000", 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050606710000000001 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1028 3.8 3:133f0075:::rbd_data.1173f77ecad8.0000000000000031:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.273575+0000", 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.099110692, 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054363240000000002, 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1028 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273575+0000", 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273575+0000", 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273574+0000", 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273579+0000", 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.566e-06 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273590+0000", 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1095e-05 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273595+0000", 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9060000000000001e-06 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273640+0000", 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5694e-05 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273777+0000", 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013655500000000001 2024-09-13T07:12:37.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.279012+0000", 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005234486 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1054 3.1e 3:7c990db8:::rbd_data.1173f77ecad8.000000000000003f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.282376+0000", 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.090309449999999999, 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049235939999999999, 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1054 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282376+0000", 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282376+0000", 2024-09-13T07:12:37.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282375+0000", 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282381+0000", 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3129999999999998e-06 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282394+0000", 2024-09-13T07:12:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3575e-05 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282402+0000", 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1049999999999995e-06 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282450+0000", 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7074999999999999e-05 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282581+0000", 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013110500000000001 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.287300+0000", 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047194280000000003 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1056 3.b 3:d41379e8:::rbd_data.1173f77ecad8.0000000000000041:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.284900+0000", 2024-09-13T07:12:37.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.087786008999999998, 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053290769999999998, 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1056 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284900+0000", 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284900+0000", 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284899+0000", 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284904+0000", 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1730000000000003e-06 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284920+0000", 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6113e-05 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284928+0000", 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7479999999999993e-06 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284973+0000", 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5024000000000001e-05 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.285115+0000", 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014217200000000001 2024-09-13T07:12:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290229+0000", 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051138709999999999 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1075 3.b 3:d41379e8:::rbd_data.1173f77ecad8.0000000000000041:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.082224953000000003, 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057904439999999996, 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:37.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290460+0000", 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290466+0000", 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290477+0000", 2024-09-13T07:12:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1595e-05 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290485+0000", 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7360000000000001e-06 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290527+0000", 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1690000000000002e-05 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290610+0000", 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3321999999999999e-05 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290800+0000", 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019058399999999999 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293341+0000", 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025410139999999999 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296160+0000", 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028181460000000001 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296209+0000", 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9055999999999999e-05 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296251+0000", 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2703000000000001e-05 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1079 3.18 3:191cddb7:::rbd_data.1173f77ecad8.0000000000000047:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.291458+0000", 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.081227964999999999, 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004881753, 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1079 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291458+0000", 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291458+0000", 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291457+0000", 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291461+0000", 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3359999999999997e-06 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291473+0000", 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909000000000001e-05 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291481+0000", 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4259999999999997e-06 2024-09-13T07:12:37.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291522+0000", 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0828000000000003e-05 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291595+0000", 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3094000000000002e-05 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291740+0000", 2024-09-13T07:12:37.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014524199999999999 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.295710+0000", 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039701650000000003 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296258+0000", 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054806600000000005 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296302+0000", 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3614e-05 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296340+0000", 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7973999999999999e-05 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1098 3.15 e25/19 3:afeba560:::rbd_data.1173f77ecad8.000000000000004f:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.073223431000000005, 2024-09-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091051389999999999, 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6639999999999998e-06 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299484+0000", 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4609e-05 2024-09-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299491+0000", 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.584e-06 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299534+0000", 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640999999999998e-05 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299553+0000", 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8564999999999999e-05 2024-09-13T07:12:37.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308532+0000", 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089789510000000006 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308568+0000", 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5896000000000003e-05 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.c epoch 25/19 rep_tid 48 entries 25'5 (0'0) error 3:32647bc6:::rbd_data.1173f77ecad8.0000000000000057:head by client.4467.0:1100 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'2)", 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.071425446000000004, 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073257039999999997, 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301266+0000", 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5239999999999997e-06 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301279+0000", 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2856e-05 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301296+0000", 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6858e-05 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301369+0000", 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2631000000000003e-05 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308586+0000", 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072174989999999996 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1104 3.1e 3:7e73e8c7:::rbd_data.1173f77ecad8.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.070823986000000005, 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009729985, 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1104 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301861+0000", 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301866+0000", 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301879+0000", 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3083e-05 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301893+0000", 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4656e-05 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301950+0000", 2024-09-13T07:12:37.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6671e-05 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302027+0000", 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6859000000000004e-05 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302260+0000", 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023318300000000001 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303132+0000", 2024-09-13T07:12:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000871487 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311492+0000", 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083604839999999996 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311550+0000", 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7701000000000002e-05 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311592+0000", 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2024000000000003e-05 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1106 3.c e25/19 3:30d6dbcc:::rbd_data.1173f77ecad8.0000000000000050:head v 25'6, mlcod=25'3)", 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.070244188999999999, 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090851709999999995, 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302441+0000", 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302447+0000", 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7969999999999997e-06 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302460+0000", 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2772000000000001e-05 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302466+0000", 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-13T07:12:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302507+0000", 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1014999999999999e-05 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302520+0000", 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2706e-05 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311485+0000", 2024-09-13T07:12:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089648929999999998 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311527+0000", 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2259000000000003e-05 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1118 3.1e 3:7d8c2ee8:::rbd_data.1173f77ecad8.000000000000005f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.063666798999999996, 2024-09-13T07:12:37.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056840500000000004, 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1118 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309018+0000", 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309023+0000", 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0950000000000003e-06 2024-09-13T07:12:37.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309034+0000", 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0857e-05 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309041+0000", 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0639999999999996e-06 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309094+0000", 2024-09-13T07:12:37.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2936000000000003e-05 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309241+0000", 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146618 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.314703+0000", 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054623320000000003 2024-09-13T07:12:37.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1117 3.c e25/19 3:32647bc6:::rbd_data.1173f77ecad8.0000000000000057:head v 25'7, mlcod=25'4)", 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.063258687999999993, 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052532430000000003, 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309433+0000", 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6730000000000003e-06 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309449+0000", 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5299999999999999e-05 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309454+0000", 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:12:37.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309495+0000", 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1285000000000002e-05 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309518+0000", 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2963e-05 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.314644+0000", 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051258409999999999 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.314680+0000", 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6650000000000003e-05 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1130 3.1e 3:7b48162d:::rbd_data.1173f77ecad8.000000000000005c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.060474607999999999, 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060295670000000004, 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1130 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312210+0000", 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312217+0000", 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.939e-06 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312229+0000", 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2269000000000001e-05 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312237+0000", 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8310000000000001e-06 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312306+0000", 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8515000000000004e-05 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312412+0000", 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312643+0000", 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023097699999999999 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.317531+0000", 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048874529999999999 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318137+0000", 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000606352 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318196+0000", 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8816000000000001e-05 2024-09-13T07:12:37.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318241+0000", 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5170999999999998e-05 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.5 epoch 25/19 rep_tid 53 entries 25'6 (0'0) error 3:a62f531c:::rbd_data.1173f77ecad8.0000000000000062:head by client.4467.0:1129 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'4)", 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.060307469000000002, 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051462469999999996, 2024-09-13T07:12:37.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312385+0000", 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0539999999999997e-06 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312396+0000", 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1208e-05 2024-09-13T07:12:37.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312404+0000", 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312453+0000", 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8915999999999997e-05 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.317525+0000", 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050714610000000002 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1154 3.b 3:d3009503:::rbd_data.1173f77ecad8.0000000000000168:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045030090000000002, 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059705729999999999, 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1154 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327655+0000", 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327661+0000", 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6860000000000003e-06 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327672+0000", 2024-09-13T07:12:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1809e-05 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327680+0000", 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9400000000000002e-06 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327729+0000", 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9023000000000001e-05 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327807+0000", 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7768999999999994e-05 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327961+0000", 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000153616 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.330853+0000", 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002891897 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333519+0000", 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026667359999999998 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333576+0000", 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6786000000000001e-05 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333626+0000", 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0144e-05 2024-09-13T07:12:37.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1155 3.b 3:d723b608:::rbd_data.1173f77ecad8.00000000000001d6:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.04454458, 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058401670000000003, 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1155 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328145+0000", 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5820000000000003e-06 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328156+0000", 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1255e-05 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328163+0000", 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0149999999999998e-06 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328502+0000", 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033881500000000001 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328582+0000", 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166000000000005e-05 2024-09-13T07:12:37.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328777+0000", 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019505799999999999 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333240+0000", 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044629070000000003 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333900+0000", 2024-09-13T07:12:37.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065926699999999995 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333943+0000", 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3027e-05 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333981+0000", 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8828999999999997e-05 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:37.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:37.471 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T07:12:37.472 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:161 writesame 0x74ed762 thru 0x74f1a69 data_size 0x68(0x4308 bytes) 2024-09-13T07:12:37.474 INFO:teuthology.orchestra.run.smithi031.stdout:162 punch from 0x92950ee to 0x9296723, (0x1635 bytes) 2024-09-13T07:12:37.474 INFO:teuthology.orchestra.run.smithi031.stdout:163 write 0x941822a thru 0x94263ae (0xe185 bytes) 2024-09-13T07:12:37.474 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.474 INFO:teuthology.orchestra.run.smithi031.stdout:165 write 0x334676a thru 0x334c24a (0x5ae1 bytes) 2024-09-13T07:12:37.474 INFO:teuthology.orchestra.run.smithi031.stdout:167 read 0x127c7d9 thru 0x128b9bb (0xf1e3 bytes) 2024-09-13T07:12:37.474 INFO:teuthology.orchestra.run.smithi031.stdout:168 trunc from 0xe992f35 to 0x51f9945 2024-09-13T07:12:37.474 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:37.473+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:37.484 INFO:teuthology.orchestra.run.smithi031.stdout:169 writesame 0x18d7633 thru 0x18e2525 data_size 0x1(0xaef3 bytes) 2024-09-13T07:12:37.510 INFO:teuthology.orchestra.run.smithi031.stdout:170 trunc from 0x51f9945 to 0x30ac946 2024-09-13T07:12:37.544 INFO:teuthology.orchestra.run.smithi031.stdout:171 writesame 0x11ecde0 thru 0x11fb6d2 data_size 0x1(0xe8f3 bytes) 2024-09-13T07:12:37.572 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:37.574+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:37.587 INFO:teuthology.orchestra.run.smithi031.stdout:173 write 0xa78e6db thru 0xa79ba46 (0xd36c bytes) 2024-09-13T07:12:37.590 INFO:teuthology.orchestra.run.smithi031.stdout:174 writesame 0x9d81288 thru 0x9d8cf28 data_size 0x1(0xbca1 bytes) 2024-09-13T07:12:37.590 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:37.590 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:37.591 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:37.591 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:37.600 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:37.614 INFO:teuthology.orchestra.run.smithi031.stdout:175 punch from 0x7f872a1 to 0x7f920a0, (0xadff bytes) 2024-09-13T07:12:37.621 INFO:teuthology.orchestra.run.smithi031.stdout:177 write 0xaf6008f thru 0xaf6a929 (0xa89b bytes) 2024-09-13T07:12:37.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.625 INFO:teuthology.orchestra.run.smithi031.stdout:183 read 0x467e967 thru 0x468087d (0x1f17 bytes) 2024-09-13T07:12:37.626 INFO:teuthology.orchestra.run.smithi031.stdout:184 read 0xabc1833 thru 0xabcec6a (0xd438 bytes) 2024-09-13T07:12:37.626 INFO:teuthology.orchestra.run.smithi031.stdout:185 punch from 0xe3a352 to 0xe49b56, (0xf804 bytes) 2024-09-13T07:12:37.626 INFO:teuthology.orchestra.run.smithi031.stdout:187 write 0x3f3235 thru 0x401309 (0xe0d5 bytes) 2024-09-13T07:12:37.632 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.633 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xe729ec5 2024-09-13T07:12:37.634 INFO:teuthology.orchestra.run.smithi031.stdout:190 trunc from 0xaf6a92a to 0xe729ec5 2024-09-13T07:12:37.639 INFO:teuthology.orchestra.run.smithi031.stdout:192 read 0xb5db9d0 thru 0xb5e2aa8 (0x70d9 bytes) 2024-09-13T07:12:37.639 INFO:teuthology.orchestra.run.smithi031.stdout:194 writesame 0x752d86a thru 0x753d705 data_size 0xa(0xfe9c bytes) 2024-09-13T07:12:37.645 INFO:teuthology.orchestra.run.smithi031.stdout:195 writesame 0x44a7f09 thru 0x44ab288 data_size 0x40(0x3380 bytes) 2024-09-13T07:12:37.647 INFO:teuthology.orchestra.run.smithi031.stdout:196 punch from 0x170fa1a to 0x171e8e4, (0xeeca bytes) 2024-09-13T07:12:37.648 INFO:teuthology.orchestra.run.smithi031.stdout:199 trunc from 0xe729ec5 to 0xdff5141 2024-09-13T07:12:37.656 INFO:teuthology.orchestra.run.smithi031.stdout:201 read 0x4a5da6a thru 0x4a6178b (0x3d22 bytes) 2024-09-13T07:12:37.656 INFO:teuthology.orchestra.run.smithi031.stdout:202 read 0x408e940 thru 0x4092466 (0x3b27 bytes) 2024-09-13T07:12:37.665 INFO:teuthology.orchestra.run.smithi031.stdout:203 trunc from 0xdff5141 to 0x5e3e97c 2024-09-13T07:12:37.673 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:37.674+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:37.682 INFO:teuthology.orchestra.run.smithi031.stdout:205 write 0x31eccb3 thru 0x31f9283 (0xc5d1 bytes) 2024-09-13T07:12:37.684 INFO:teuthology.orchestra.run.smithi031.stdout:206 writesame 0x170cc61 thru 0x171693a data_size 0x11(0x9cda bytes) 2024-09-13T07:12:37.687 INFO:teuthology.orchestra.run.smithi031.stdout:207 writesame 0x41e0562 thru 0x41e569a data_size 0x1d(0x5139 bytes) 2024-09-13T07:12:37.692 INFO:teuthology.orchestra.run.smithi031.stdout:211 write 0x4569660 thru 0x4576536 (0xced7 bytes) 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:213 punch from 0x38c51ac to 0x38c5f53, (0xda7 bytes) 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:215 read 0x5028ed2 thru 0x502ec5d (0x5d8c bytes) 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:220 read 0x3fbb273 thru 0x3fc43f3 (0x9181 bytes) 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:224 read 0x4713bbd thru 0x471f541 (0xb985 bytes) 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:225 read 0x5742a71 thru 0x57506fd (0xdc8d bytes) 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:227 punch from 0x45d87c6 to 0x45e5445, (0xcc7f bytes) 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.698 INFO:teuthology.orchestra.run.smithi031.stdout:232 punch from 0x45488fa to 0x4554d29, (0xc42f bytes) 2024-09-13T07:12:37.699 INFO:teuthology.orchestra.run.smithi031.stdout:233 punch from 0x23ffd01 to 0x2407f9e, (0x829d bytes) 2024-09-13T07:12:37.699 INFO:teuthology.orchestra.run.smithi031.stdout:234 trunc from 0x5e3e97c to 0x5224377 2024-09-13T07:12:37.704 INFO:teuthology.orchestra.run.smithi031.stdout:235 writesame 0xf6be7e thru 0xf6fee1 data_size 0x1a(0x4064 bytes) 2024-09-13T07:12:37.707 INFO:teuthology.orchestra.run.smithi031.stdout:236 read 0x4a2f80 thru 0x4aabef (0x7c70 bytes) 2024-09-13T07:12:37.711 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:37.711 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:37.711 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:37.711 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:37.712 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:37.712 INFO:teuthology.orchestra.run.smithi031.stdout:237 trunc from 0x5224377 to 0x9becaad 2024-09-13T07:12:37.721 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:37.729 INFO:teuthology.orchestra.run.smithi031.stdout:238 write 0xccc73e9 thru 0xccd5c81 (0xe899 bytes) 2024-09-13T07:12:37.733 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.733 INFO:teuthology.orchestra.run.smithi031.stdout:240 punch from 0x3196410 to 0x319fee0, (0x9ad0 bytes) 2024-09-13T07:12:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:242 trunc from 0xccd5c82 to 0x60f61f4 2024-09-13T07:12:37.756 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.756 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.760 INFO:teuthology.orchestra.run.smithi031.stdout:248 trunc from 0x60f61f4 to 0x20673f0 2024-09-13T07:12:37.774 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:37.775+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:37.796 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.796 INFO:teuthology.orchestra.run.smithi031.stdout:251 punch from 0x128884 to 0x136b4a, (0xe2c6 bytes) 2024-09-13T07:12:37.797 INFO:teuthology.orchestra.run.smithi031.stdout:252 punch from 0xf8821f to 0xf95bac, (0xd98d bytes) 2024-09-13T07:12:37.802 INFO:teuthology.orchestra.run.smithi031.stdout:253 trunc from 0x20673f0 to 0x6e07749 2024-09-13T07:12:37.807 INFO:teuthology.orchestra.run.smithi031.stdout:255 write 0x8f5ab9 thru 0x901c1e (0xc166 bytes) 2024-09-13T07:12:37.812 INFO:teuthology.orchestra.run.smithi031.stdout:256 read 0x6b2c57c thru 0x6b2e896 (0x231b bytes) 2024-09-13T07:12:37.813 INFO:teuthology.orchestra.run.smithi031.stdout:257 punch from 0x441b73d to 0x441d51e, (0x1de1 bytes) 2024-09-13T07:12:37.813 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.813 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.813 INFO:teuthology.orchestra.run.smithi031.stdout:260 punch from 0x63eeb61 to 0x63f99ea, (0xae89 bytes) 2024-09-13T07:12:37.821 INFO:teuthology.orchestra.run.smithi031.stdout:262 write 0x90cec9b thru 0x90d86ea (0x9a50 bytes) 2024-09-13T07:12:37.824 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:264 trunc from 0x90d86eb to 0x2f29a8c 2024-09-13T07:12:37.841 INFO:teuthology.orchestra.run.smithi031.stdout:265 read 0x2bc1dd4 thru 0x2bcb1a3 (0x93d0 bytes) 2024-09-13T07:12:37.841 INFO:teuthology.orchestra.run.smithi031.stdout:266 punch from 0x21c59f3 to 0x21cf0f0, (0x96fd bytes) 2024-09-13T07:12:37.860 INFO:teuthology.orchestra.run.smithi031.stdout:268 write 0xeba7e4a thru 0xeba7e9e (0x55 bytes) 2024-09-13T07:12:37.863 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.864 INFO:teuthology.orchestra.run.smithi031.stdout:273 read 0x989bb99 thru 0x98aaba7 (0xf00f bytes) 2024-09-13T07:12:37.864 INFO:teuthology.orchestra.run.smithi031.stdout:275 punch from 0xdb67fe8 to 0xdb6dcf8, (0x5d10 bytes) 2024-09-13T07:12:37.864 INFO:teuthology.orchestra.run.smithi031.stdout:276 read 0xe53de6 thru 0xe5cd87 (0x8fa2 bytes) 2024-09-13T07:12:37.864 INFO:teuthology.orchestra.run.smithi031.stdout:277 writesame 0xaa05680 thru 0xaa0fdd6 data_size 0x1(0xa757 bytes) 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:791 3.14 3:2ad4b4d4:::rbd_data.1173f77ecad8.0000000000000147:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:36.639691+0000", 2024-09-13T07:12:37.876 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.236249403, 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064195340000000002, 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 791 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639691+0000", 2024-09-13T07:12:37.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639691+0000", 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639690+0000", 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639693+0000", 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.399e-06 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639703+0000", 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.611e-06 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639708+0000", 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4060000000000004e-06 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639740+0000", 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2020999999999997e-05 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639821+0000", 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1137000000000006e-05 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:37.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.640784+0000", 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00096226000000000005 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.645971+0000", 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051877240000000003 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.646012+0000", 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0537999999999997e-05 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.646062+0000", 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0489999999999999e-05 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.646110+0000", 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7574000000000002e-05 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:914 3.14 3:2ed78eca:::rbd_data.1173f77ecad8.000000000000000a:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:37.882 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.62951926199999997, 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082734499999999999, 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 914 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246420+0000", 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246424+0000", 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246430+0000", 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8000000000000001e-06 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246434+0000", 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246477+0000", 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2443e-05 2024-09-13T07:12:37.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246632+0000", 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015543100000000001 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254694+0000", 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080618979999999996 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:919 3.1a 3:5d4b50e3:::rbd_data.1173f77ecad8.000000000000000d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.248431+0000", 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.627509125, 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060058289999999999, 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 919 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248431+0000", 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248431+0000", 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248430+0000", 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248434+0000", 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2089999999999999e-06 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248446+0000", 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1715999999999999e-05 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248454+0000", 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3790000000000006e-06 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248482+0000", 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7778999999999999e-05 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248554+0000", 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2334000000000002e-05 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254437+0000", 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058821510000000004 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "pg_update_log_missing(3.1 epoch 25/19 rep_tid 36 entries 25'20 (0'0) error 3:87900178:::rbd_data.1173f77ecad8.000000000000000e:head by client.4467.0:920 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'18)", 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.248944+0000", 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.62699640400000001, 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054260560000000003, 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:12:37.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248944+0000", 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248944+0000", 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248943+0000", 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248950+0000", 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248963+0000", 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2836e-05 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248977+0000", 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4039999999999999e-05 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.249018+0000", 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1362999999999997e-05 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254370+0000", 2024-09-13T07:12:37.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005351682 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "pg_update_log_missing(3.a epoch 25/19 rep_tid 12 entries 25'10 (0'0) error 3:52856a69:::rbd_data.1173f77ecad8.000000000000000f:head by client.4467.0:922 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'8)", 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.251656+0000", 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.62428375400000002, 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053679449999999998, 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251656+0000", 2024-09-13T07:12:37.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251656+0000", 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251655+0000", 2024-09-13T07:12:37.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251665+0000", 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0860000000000008e-06 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251682+0000", 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7102e-05 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251693+0000", 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1259e-05 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251747+0000", 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4239e-05 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.257024+0000", 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052769679999999999 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1103 3.9 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.301886+0000", 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.57405420600000001, 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053560300000000003, 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1103 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301886+0000", 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301886+0000", 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301885+0000", 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301889+0000", 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5040000000000004e-06 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301901+0000", 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.189e-05 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301908+0000", 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1290000000000001e-06 2024-09-13T07:12:37.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301947+0000", 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9629999999999998e-05 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.301995+0000", 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.812e-05 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.302109+0000", 2024-09-13T07:12:37.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000113663 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.305883+0000", 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037737869999999998 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.307191+0000", 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001308556 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.307219+0000", 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7979000000000001e-05 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.307242+0000", 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2521999999999999e-05 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1128 3.3 3:c2b5a8ac:::rbd_data.1173f77ecad8.0000000000000061:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.311928+0000", 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.56401209699999999, 2024-09-13T07:12:37.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061792139999999997, 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1128 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311928+0000", 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311928+0000", 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311927+0000", 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311932+0000", 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311943+0000", 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0625000000000001e-05 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311954+0000", 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679000000000001e-05 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311998+0000", 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.401e-05 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312088+0000", 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0781999999999999e-05 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.318107+0000", 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060187640000000002 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1151 3.9 3:9280c38f:::rbd_data.1173f77ecad8.000000000000010c:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:37.908 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.54907925300000004, 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069283519999999996, 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1151 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:37.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326860+0000", 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326863+0000", 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326872+0000", 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2199999999999992e-06 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326886+0000", 2024-09-13T07:12:37.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4066999999999999e-05 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327486+0000", 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00060069599999999996 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327560+0000", 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3709999999999997e-05 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:37.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327669+0000", 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108544 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.330996+0000", 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033276099999999999 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333730+0000", 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002733311 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333767+0000", 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7861000000000002e-05 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333789+0000", 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1608999999999999e-05 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1153 3.1b e25/19 3:d88532a6:::rbd_data.1173f77ecad8.000000000000015c:head v 25'14, mlcod=25'11)", 2024-09-13T07:12:37.914 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.54837239699999996, 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091902019999999997, 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327567+0000", 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327572+0000", 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327579+0000", 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.306e-06 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327585+0000", 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1279999999999996e-06 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327625+0000", 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9227999999999998e-05 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327641+0000", 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6059999999999999e-05 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336708+0000", 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090672949999999995 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336758+0000", 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9870000000000002e-05 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.918 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1530 3.14 3:2eb924b3:::rbd_data.1173c53667c.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.764210+0000", 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.111729939, 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052332680000000001, 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1530 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764210+0000", 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764210+0000", 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764209+0000", 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764215+0000", 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7950000000000004e-06 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764229+0000", 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3987e-05 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764237+0000", 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9729999999999997e-06 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764304+0000", 2024-09-13T07:12:37.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7570000000000005e-05 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764395+0000", 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0957000000000003e-05 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.764504+0000", 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010881 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:37.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766397+0000", 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001892464 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769363+0000", 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002966392 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769403+0000", 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0358000000000002e-05 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769443+0000", 2024-09-13T07:12:37.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9867999999999998e-05 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1537 3.13 e25/19 3:cb72c382:::rbd_data.1173c53667c.0000000000000038:head v 25'13, mlcod=25'11)", 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.766482+0000", 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.109458412, 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053468509999999997, 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766482+0000", 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766482+0000", 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766481+0000", 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766487+0000", 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5180000000000001e-06 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766501+0000", 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3439e-05 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766508+0000", 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766553+0000", 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5039e-05 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766573+0000", 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0032999999999999e-05 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.771794+0000", 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052208580000000001 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.771828+0000", 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4700000000000003e-05 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1551 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'89, mlcod=25'86)", 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.097715094000000002, 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077485289999999997, 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778224+0000", 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778233+0000", 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4689999999999997e-06 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778245+0000", 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2097e-05 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778252+0000", 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.934e-06 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778288+0000", 2024-09-13T07:12:37.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6012e-05 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778302+0000", 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4630999999999999e-05 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785938+0000", 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076354329999999996 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785973+0000", 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5663000000000001e-05 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1552 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'90, mlcod=25'86)", 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.097066828999999993, 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0071221499999999998, 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.933 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778887+0000", 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4409999999999999e-05 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778897+0000", 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0344e-05 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778903+0000", 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7019999999999998e-06 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778933+0000", 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0358e-05 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778955+0000", 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1795999999999999e-05 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785976+0000", 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070212269999999997 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785995+0000", 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8907e-05 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.937 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1553 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'91, mlcod=25'86)", 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.779472+0000", 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.096468031999999995, 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065460170000000003, 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779472+0000", 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779472+0000", 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779471+0000", 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779475+0000", 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.07e-06 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779482+0000", 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8850000000000001e-06 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779487+0000", 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6090000000000003e-06 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779513+0000", 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5843e-05 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779530+0000", 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6889e-05 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785997+0000", 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064669389999999997 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786018+0000", 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1335000000000002e-05 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1554 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'92, mlcod=25'86)", 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.780024+0000", 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.095916219999999996, 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060148739999999999, 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780024+0000", 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780024+0000", 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780023+0000", 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780029+0000", 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5840000000000002e-06 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780039+0000", 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0743e-05 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780045+0000", 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9120000000000003e-06 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780078+0000", 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2286999999999998e-05 2024-09-13T07:12:37.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780094+0000", 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6041000000000001e-05 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786019+0000", 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059255449999999999 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786039+0000", 2024-09-13T07:12:37.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9403999999999999e-05 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1555 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'93, mlcod=25'86)", 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.095283007000000003, 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0081821140000000007, 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780656+0000", 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780661+0000", 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.079e-06 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780670+0000", 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6030000000000005e-06 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780676+0000", 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780720+0000", 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3767000000000003e-05 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780739+0000", 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9504000000000002e-05 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788759+0000", 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080197340000000006 2024-09-13T07:12:37.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788839+0000", 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0075999999999997e-05 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1556 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'94, mlcod=25'86)", 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.094606249000000003, 2024-09-13T07:12:37.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0075812830000000003, 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781333+0000", 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781337+0000", 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781344+0000", 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781349+0000", 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781376+0000", 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7826000000000001e-05 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781392+0000", 2024-09-13T07:12:37.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5401999999999999e-05 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788842+0000", 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007450684 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788915+0000", 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2556000000000001e-05 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1557 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'95, mlcod=25'86)", 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.094015016000000007, 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070583310000000002, 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:37.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781928+0000", 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4699999999999998e-06 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781934+0000", 2024-09-13T07:12:37.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.302e-06 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781938+0000", 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2559999999999998e-06 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781967+0000", 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9856e-05 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781979+0000", 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1857e-05 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788917+0000", 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069380750000000001 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788983+0000", 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5906000000000002e-05 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1558 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'96, mlcod=25'87)", 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.782560+0000", 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.093380001000000004, 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064554690000000001, 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782560+0000", 2024-09-13T07:12:37.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782560+0000", 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782559+0000", 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782564+0000", 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8289999999999998e-06 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782571+0000", 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0679999999999999e-06 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782578+0000", 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7190000000000004e-06 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782612+0000", 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3918000000000003e-05 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782623+0000", 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1326e-05 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788985+0000", 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0063617020000000003 2024-09-13T07:12:37.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789015+0000", 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0454999999999999e-05 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1559 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'97, mlcod=25'87)", 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.783107+0000", 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.092832977999999997, 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059300250000000002, 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:37.963 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783107+0000", 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783107+0000", 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:37.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783106+0000", 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783110+0000", 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8249999999999998e-06 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783117+0000", 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:37.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783121+0000", 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7979999999999999e-06 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783151+0000", 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9873e-05 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783161+0000", 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0403e-05 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:37.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789017+0000", 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058556729999999996 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789037+0000", 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.014e-05 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:37.967 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:37.968 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:37.876+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:37.969 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:280 write 0xb2fd153 thru 0xb306292 (0x9140 bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:282 read 0xa65358 thru 0xa73508 (0xe1b1 bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:284 punch from 0x63bbf54 to 0x63c5aef, (0x9b9b bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:285 punch from 0xb2e1abf to 0xb2e3cbb, (0x21fc bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:286 read 0x766d085 thru 0x7678b7a (0xbaf6 bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:287 writesame 0x1c07f72 thru 0x1c17c14 data_size 0x41(0xfca3 bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:288 writesame 0x9c530d4 thru 0x9c5c0bc data_size 0x1(0x8fe9 bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:289 punch from 0x121f0c6 to 0x122477b, (0x56b5 bytes) 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:291 write 0x4ad0b60 thru 0x4adc5c3 (0xba64 bytes) 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:292 trunc from 0xeba7e9f to 0x7bf3c3e 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:297 punch from 0x2c7348c to 0x2c78e9c, (0x5a10 bytes) 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:298 trunc from 0x7bf3c3e to 0xf2343 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:300 read 0x2c882 thru 0x35108 (0x8887 bytes) 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:301 punch from 0xd76db to 0xdd37f, (0x5ca4 bytes) 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.972 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:305 writesame 0x4daade1 thru 0x4dafe60 data_size 0x8(0x5080 bytes) 2024-09-13T07:12:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:307 read 0x4577d07 thru 0x457ebbd (0x6eb7 bytes) 2024-09-13T07:12:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:309 write 0x16b73ac thru 0x16bd497 (0x60ec bytes) 2024-09-13T07:12:37.976 INFO:teuthology.orchestra.run.smithi031.stdout:311 read 0x3cd7315 thru 0x3cdeea4 (0x7b90 bytes) 2024-09-13T07:12:37.984 INFO:teuthology.orchestra.run.smithi031.stdout:312 trunc from 0x4dafe61 to 0xc703634 2024-09-13T07:12:37.990 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.990 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:37.990 INFO:teuthology.orchestra.run.smithi031.stdout:315 writesame 0x2669669 thru 0x2675ecd data_size 0x1d(0xc865 bytes) 2024-09-13T07:12:37.993 INFO:teuthology.orchestra.run.smithi031.stdout:317 write 0x4a25a39 thru 0x4a269a3 (0xf6b bytes) 2024-09-13T07:12:37.997 INFO:teuthology.orchestra.run.smithi031.stdout:319 write 0x876b81e thru 0x87761f9 (0xa9dc bytes) 2024-09-13T07:12:38.001 INFO:teuthology.orchestra.run.smithi031.stdout:320 write 0x791efb3 thru 0x792a989 (0xb9d7 bytes) 2024-09-13T07:12:38.011 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:38.012+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:38.011 INFO:teuthology.orchestra.run.smithi031.stdout:321 write 0xe10d8fd thru 0xe116dd7 (0x94db bytes) 2024-09-13T07:12:38.015 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.015 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.022 INFO:teuthology.orchestra.run.smithi031.stdout:324 write 0xe85a9fe thru 0xe85d1b7 (0x27ba bytes) 2024-09-13T07:12:38.027 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.027 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.027 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.027 INFO:teuthology.orchestra.run.smithi031.stdout:329 punch from 0x434c642 to 0x435309b, (0x6a59 bytes) 2024-09-13T07:12:38.027 INFO:teuthology.orchestra.run.smithi031.stdout:330 read 0x4da971c thru 0x4dac553 (0x2e38 bytes) 2024-09-13T07:12:38.027 INFO:teuthology.orchestra.run.smithi031.stdout:331 write 0x6f20e18 thru 0x6f29d40 (0x8f29 bytes) 2024-09-13T07:12:38.030 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.030 INFO:teuthology.orchestra.run.smithi031.stdout:335 write 0xba88b2a thru 0xba988ce (0xfda5 bytes) 2024-09-13T07:12:38.033 INFO:teuthology.orchestra.run.smithi031.stdout:338 writesame 0x5e94167 thru 0x5ea391b data_size 0x1(0xf7b5 bytes) 2024-09-13T07:12:38.060 INFO:teuthology.orchestra.run.smithi031.stdout:339 read 0xc0258d7 thru 0xc02ac4c (0x5376 bytes) 2024-09-13T07:12:38.060 INFO:teuthology.orchestra.run.smithi031.stdout:340 write 0x292b39f thru 0x2930748 (0x53aa bytes) 2024-09-13T07:12:38.064 INFO:teuthology.orchestra.run.smithi031.stdout:342 punch from 0xc1cb873 to 0xc1cf385, (0x3b12 bytes) 2024-09-13T07:12:38.064 INFO:teuthology.orchestra.run.smithi031.stdout:343 read 0x3a40542 thru 0x3a4e7dc (0xe29b bytes) 2024-09-13T07:12:38.064 INFO:teuthology.orchestra.run.smithi031.stdout:347 read 0xcac98d9 thru 0xcacbb64 (0x228c bytes) 2024-09-13T07:12:38.064 INFO:teuthology.orchestra.run.smithi031.stdout:349 write 0x2fa6e04 thru 0x2fa7515 (0x712 bytes) 2024-09-13T07:12:38.069 INFO:teuthology.orchestra.run.smithi031.stdout:352 write 0x225f05f thru 0x2266782 (0x7724 bytes) 2024-09-13T07:12:38.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:12:38.072 INFO:teuthology.orchestra.run.smithi031.stdout:353 read 0x24a1fe7 thru 0x24a24c0 (0x4da bytes) 2024-09-13T07:12:38.072 INFO:teuthology.orchestra.run.smithi031.stdout:354 writesame 0xafec38f thru 0xafeebdd data_size 0x1(0x284f bytes) 2024-09-13T07:12:38.078 INFO:teuthology.orchestra.run.smithi031.stdout:357 write 0xb01ee83 thru 0xb02d3e7 (0xe565 bytes) 2024-09-13T07:12:38.079 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:38.079+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:38.081 INFO:teuthology.orchestra.run.smithi031.stdout:358 writesame 0x3138049 thru 0x313ba86 data_size 0x2a(0x3a3e bytes) 2024-09-13T07:12:38.084 INFO:teuthology.orchestra.run.smithi031.stdout:363 write 0x2c92223 thru 0x2c9d662 (0xb440 bytes) 2024-09-13T07:12:38.087 INFO:teuthology.orchestra.run.smithi031.stdout:365 read 0xc59dd44 thru 0xc5a86df (0xa99c bytes) 2024-09-13T07:12:38.087 INFO:teuthology.orchestra.run.smithi031.stdout:366 write 0xe712a91 thru 0xe71af70 (0x84e0 bytes) 2024-09-13T07:12:38.091 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.091 INFO:teuthology.orchestra.run.smithi031.stdout:369 read 0x3994ba7 thru 0x39952d4 (0x72e bytes) 2024-09-13T07:12:38.092 INFO:teuthology.orchestra.run.smithi031.stdout:370 read 0xe0bc8a9 thru 0xe0bdb43 (0x129b bytes) 2024-09-13T07:12:38.092 INFO:teuthology.orchestra.run.smithi031.stdout:371 punch from 0x4f5bc5d to 0x4f6909a, (0xd43d bytes) 2024-09-13T07:12:38.104 INFO:teuthology.orchestra.run.smithi031.stdout:372 trunc from 0xe85d1b8 to 0x326f229 2024-09-13T07:12:38.121 INFO:teuthology.orchestra.run.smithi031.stdout:373 trunc from 0x326f229 to 0x77d75c4 2024-09-13T07:12:38.124 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:38.124 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:38.124 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:38.124 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:38.132 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:12:38.134 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:38.142 INFO:teuthology.orchestra.run.smithi031.stdout:374 write 0xc06b3f8 thru 0xc06f82c (0x4435 bytes) 2024-09-13T07:12:38.153 INFO:teuthology.orchestra.run.smithi031.stdout:375 read 0x8b7ed6c thru 0x8b8e6b3 (0xf948 bytes) 2024-09-13T07:12:38.153 INFO:teuthology.orchestra.run.smithi031.stdout:377 read 0xaa7451a thru 0xaa75fdf (0x1ac6 bytes) 2024-09-13T07:12:38.153 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.154 INFO:teuthology.orchestra.run.smithi031.stdout:382 trunc from 0xc06f82d to 0xcf8dafe 2024-09-13T07:12:38.164 INFO:teuthology.orchestra.run.smithi031.stdout:383 write 0xd49e429 thru 0xd4a6a01 (0x85d9 bytes) 2024-09-13T07:12:38.168 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.175 INFO:teuthology.orchestra.run.smithi031.stdout:386 trunc from 0xd4a6a02 to 0x741688c 2024-09-13T07:12:38.177 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:38.178+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:38.186 INFO:teuthology.orchestra.run.smithi031.stdout:387 punch from 0x70b4ad4 to 0x70b554a, (0xa76 bytes) 2024-09-13T07:12:38.186 INFO:teuthology.orchestra.run.smithi031.stdout:388 read 0x18fe176 thru 0x1906449 (0x82d4 bytes) 2024-09-13T07:12:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:389 write 0xaa8a67b thru 0xaa959c2 (0xb348 bytes) 2024-09-13T07:12:38.205 INFO:teuthology.orchestra.run.smithi031.stdout:390 trunc from 0xaa959c3 to 0x4a3a4b6 2024-09-13T07:12:38.210 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:38.214 INFO:teuthology.orchestra.run.smithi031.stdout:391 read 0x41441b0 thru 0x41479fe (0x384f bytes) 2024-09-13T07:12:38.214 INFO:teuthology.orchestra.run.smithi031.stdout:392 read 0x467a4ec thru 0x4689b61 (0xf676 bytes) 2024-09-13T07:12:38.228 INFO:teuthology.orchestra.run.smithi031.stdout:393 writesame 0xc40c45c thru 0xc41a1e2 data_size 0x1(0xdd87 bytes) 2024-09-13T07:12:38.228 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:38.229 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:38.235 INFO:teuthology.orchestra.run.smithi031.stdout:394 write 0xe484222 thru 0xe48f52d (0xb30c bytes) 2024-09-13T07:12:38.239 INFO:teuthology.orchestra.run.smithi031.stdout:395 write 0xa3f8bf7 thru 0xa3fb18f (0x2599 bytes) 2024-09-13T07:12:38.241 INFO:teuthology.orchestra.run.smithi031.stdout:396 write 0x6a73901 thru 0x6a76d4d (0x344d bytes) 2024-09-13T07:12:38.244 INFO:teuthology.orchestra.run.smithi031.stdout:397 read 0xace4f23 thru 0xacefc6e (0xad4c bytes) 2024-09-13T07:12:38.244 INFO:teuthology.orchestra.run.smithi031.stdout:399 read 0x4b46455 thru 0x4b4c02f (0x5bdb bytes) 2024-09-13T07:12:38.244 INFO:teuthology.orchestra.run.smithi031.stdout:400 punch from 0x2a8b84a to 0x2a8d345, (0x1afb bytes) 2024-09-13T07:12:38.244 INFO:teuthology.orchestra.run.smithi031.stdout:401 writesame 0x20047e0 thru 0x20077c9 data_size 0x1(0x2fea bytes) 2024-09-13T07:12:38.246 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:38.246 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:38.248 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:38.249 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:38.249 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:38.249 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:38.249 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:38.252 INFO:teuthology.orchestra.run.smithi031.stdout:403 writesame 0x7ba860e thru 0x7baed3e data_size 0x1(0x6731 bytes) 2024-09-13T07:12:38.258 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:38.262 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:38.267 INFO:teuthology.orchestra.run.smithi031.stdout:405 write 0x454b381 thru 0x4550d2f (0x59af bytes) 2024-09-13T07:12:38.275 INFO:teuthology.orchestra.run.smithi031.stdout:406 trunc from 0xe48f52e to 0x9cade1c 2024-09-13T07:12:38.282 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:38.287 INFO:teuthology.orchestra.run.smithi031.stdout:408 punch from 0x4eb8e0c to 0x4ebd750, (0x4944 bytes) 2024-09-13T07:12:38.287 INFO:teuthology.orchestra.run.smithi031.stdout:410 read 0x6e122c thru 0x6e8917 (0x76ec bytes) 2024-09-13T07:12:38.297 INFO:teuthology.orchestra.run.smithi031.stdout:411 write 0xce01333 thru 0xce01cbe (0x98c bytes) 2024-09-13T07:12:38.301 INFO:teuthology.orchestra.run.smithi031.stdout:412 write 0x9bc5bcd thru 0x9bc7aa3 (0x1ed7 bytes) 2024-09-13T07:12:38.301 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:38.301 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:38.302 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:38.303+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:38.310 INFO:teuthology.orchestra.run.smithi031.stdout:417 write 0xd3122b5 thru 0xd31fbe3 (0xd92f bytes) 2024-09-13T07:12:38.314 INFO:teuthology.orchestra.run.smithi031.stdout:418 read 0x80ca02c thru 0x80cf370 (0x5345 bytes) 2024-09-13T07:12:38.314 INFO:teuthology.orchestra.run.smithi031.stdout:419 punch from 0x82e25ac to 0x82e7257, (0x4cab bytes) 2024-09-13T07:12:38.314 INFO:teuthology.orchestra.run.smithi031.stdout:421 punch from 0xc1b6b0c to 0xc1c190d, (0xae01 bytes) 2024-09-13T07:12:38.314 INFO:teuthology.orchestra.run.smithi031.stdout:422 writesame 0x631578c thru 0x631aaa8 data_size 0x1(0x531d bytes) 2024-09-13T07:12:38.319 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:38.325 INFO:teuthology.orchestra.run.smithi031.stdout:424 read 0xbcf0bab thru 0xbcf8353 (0x77a9 bytes) 2024-09-13T07:12:38.325 INFO:teuthology.orchestra.run.smithi031.stdout:425 write 0x2f2df27 thru 0x2f36332 (0x840c bytes) 2024-09-13T07:12:38.336 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:12:38.337 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:12:38.339 INFO:teuthology.orchestra.run.smithi031.stdout:427 trunc from 0xd31fbe4 to 0x3351ce0 2024-09-13T07:12:38.363 INFO:teuthology.orchestra.run.smithi031.stdout:428 trunc from 0x3351ce0 to 0xafc45d6 2024-09-13T07:12:38.368 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:38.369 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:38.369 INFO:teuthology.orchestra.run.smithi031.stdout:429 writesame 0xa2daef9 thru 0xa2e7bf9 data_size 0xd(0xcd01 bytes) 2024-09-13T07:12:38.373 INFO:teuthology.orchestra.run.smithi031.stdout:430 read 0x50891f0 thru 0x508e799 (0x55aa bytes) 2024-09-13T07:12:38.378 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:38.380 INFO:teuthology.orchestra.run.smithi031.stdout:431 write 0xb663b07 thru 0xb66ca74 (0x8f6e bytes) 2024-09-13T07:12:38.382 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.383 INFO:teuthology.orchestra.run.smithi031.stdout:433 write 0x49e0fd2 thru 0x49e3158 (0x2187 bytes) 2024-09-13T07:12:38.388 INFO:teuthology.orchestra.run.smithi031.stdout:434 write 0x2a8e2c0 thru 0x2a9bfaf (0xdcf0 bytes) 2024-09-13T07:12:38.391 INFO:teuthology.orchestra.run.smithi031.stdout:436 punch from 0x1ad5b42 to 0x1ad693d, (0xdfb bytes) 2024-09-13T07:12:38.391 INFO:teuthology.orchestra.run.smithi031.stdout:438 read 0x248c488 thru 0x248f4d9 (0x3052 bytes) 2024-09-13T07:12:38.391 INFO:teuthology.orchestra.run.smithi031.stdout:439 read 0x2340d07 thru 0x234daab (0xcda5 bytes) 2024-09-13T07:12:38.391 INFO:teuthology.orchestra.run.smithi031.stdout:440 write 0x8d91149 thru 0x8d9531e (0x41d6 bytes) 2024-09-13T07:12:38.394 INFO:teuthology.orchestra.run.smithi031.stdout:441 write 0x1533532 thru 0x1535595 (0x2064 bytes) 2024-09-13T07:12:38.401 INFO:teuthology.orchestra.run.smithi031.stdout:442 trunc from 0xb66ca75 to 0xde629db 2024-09-13T07:12:38.407 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.407 INFO:teuthology.orchestra.run.smithi031.stdout:444 write 0x4bd056 thru 0x4cc8b0 (0xf85b bytes) 2024-09-13T07:12:38.409 INFO:teuthology.orchestra.run.smithi031.stdout:445 punch from 0x2369a2c to 0x237088e, (0x6e62 bytes) 2024-09-13T07:12:38.410 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.410 INFO:teuthology.orchestra.run.smithi031.stdout:447 read 0xa3538d6 thru 0xa357220 (0x394b bytes) 2024-09-13T07:12:38.410 INFO:teuthology.orchestra.run.smithi031.stdout:448 write 0x6c30cbd thru 0x6c3c1ed (0xb531 bytes) 2024-09-13T07:12:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:449 punch from 0x9cbb078 to 0x9cc137f, (0x6307 bytes) 2024-09-13T07:12:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:450 write 0xc705018 thru 0xc70c53a (0x7523 bytes) 2024-09-13T07:12:38.418 INFO:teuthology.orchestra.run.smithi031.stdout:451 writesame 0x4df8bec thru 0x4dfa316 data_size 0x3(0x172b bytes) 2024-09-13T07:12:38.419 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:38.420+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:38.422 INFO:teuthology.orchestra.run.smithi031.stdout:452 read 0x43072a3 thru 0x430fb9c (0x88fa bytes) 2024-09-13T07:12:38.422 INFO:teuthology.orchestra.run.smithi031.stdout:453 write 0xcc63bbf thru 0xcc6ef3e (0xb380 bytes) 2024-09-13T07:12:38.425 INFO:teuthology.orchestra.run.smithi031.stdout:455 write 0x20d0390 thru 0x20d3f32 (0x3ba3 bytes) 2024-09-13T07:12:38.428 INFO:teuthology.orchestra.run.smithi031.stdout:456 write 0xcf52a2 thru 0xd01c1c (0xc97b bytes) 2024-09-13T07:12:38.434 INFO:teuthology.orchestra.run.smithi031.stdout:457 write 0x7308122 thru 0x730b366 (0x3245 bytes) 2024-09-13T07:12:38.437 INFO:teuthology.orchestra.run.smithi031.stdout:458 write 0xe69d41 thru 0xe775e9 (0xd8a9 bytes) 2024-09-13T07:12:38.440 INFO:teuthology.orchestra.run.smithi031.stdout:459 write 0x528fbbb thru 0x529b12c (0xb572 bytes) 2024-09-13T07:12:38.442 INFO:teuthology.orchestra.run.smithi031.stdout:460 write 0x2b9c5b4 thru 0x2ba0657 (0x40a4 bytes) 2024-09-13T07:12:38.448 INFO:teuthology.orchestra.run.smithi031.stdout:462 write 0x3095d40 thru 0x3099ffa (0x42bb bytes) 2024-09-13T07:12:38.448 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.449 INFO:teuthology.orchestra.run.smithi031.stdout:464 writesame 0x2159a0d thru 0x215f091 data_size 0x45(0x5685 bytes) 2024-09-13T07:12:38.450 INFO:teuthology.orchestra.run.smithi031.stdout:465 write 0xc66ba19 thru 0xc677d88 (0xc370 bytes) 2024-09-13T07:12:38.453 INFO:teuthology.orchestra.run.smithi031.stdout:466 write 0x3ceeecd thru 0x3cf7fdb (0x910f bytes) 2024-09-13T07:12:38.456 INFO:teuthology.orchestra.run.smithi031.stdout:467 writesame 0x492fd6f thru 0x49372e0 data_size 0x3(0x7572 bytes) 2024-09-13T07:12:38.461 INFO:teuthology.orchestra.run.smithi031.stdout:468 writesame 0x38d4f1a thru 0x38de27a data_size 0x1(0x9361 bytes) 2024-09-13T07:12:38.481 INFO:teuthology.orchestra.run.smithi031.stdout:469 read 0x2888971 thru 0x288ff3e (0x75ce bytes) 2024-09-13T07:12:38.481 INFO:teuthology.orchestra.run.smithi031.stdout:470 punch from 0xc6dddbc to 0xc6ec8bb, (0xeaff bytes) 2024-09-13T07:12:38.481 INFO:teuthology.orchestra.run.smithi031.stdout:471 writesame 0x5a082cd thru 0x5a0d343 data_size 0x1(0x5077 bytes) 2024-09-13T07:12:38.481 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:38.482+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:38.492 INFO:teuthology.orchestra.run.smithi031.stdout:472 punch from 0x1ee74c4 to 0x1eeb9e1, (0x451d bytes) 2024-09-13T07:12:38.492 INFO:teuthology.orchestra.run.smithi031.stdout:473 read 0xc1e3b58 thru 0xc1ef176 (0xb61f bytes) 2024-09-13T07:12:38.492 INFO:teuthology.orchestra.run.smithi031.stdout:474 read 0x372cc80 thru 0x372f192 (0x2513 bytes) 2024-09-13T07:12:38.492 INFO:teuthology.orchestra.run.smithi031.stdout:475 write 0xc6dfb21 thru 0xc6ea1bc (0xa69c bytes) 2024-09-13T07:12:38.496 INFO:teuthology.orchestra.run.smithi031.stdout:476 punch from 0x21908e7 to 0x219d59e, (0xccb7 bytes) 2024-09-13T07:12:38.501 INFO:teuthology.orchestra.run.smithi031.stdout:477 write 0xe442857 thru 0xe44731a (0x4ac4 bytes) 2024-09-13T07:12:38.504 INFO:teuthology.orchestra.run.smithi031.stdout:479 write 0xb9baf89 thru 0xb9c9f6b (0xefe3 bytes) 2024-09-13T07:12:38.507 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:482 read 0x3a40f81 thru 0x3a4dff3 (0xd073 bytes) 2024-09-13T07:12:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:485 read 0x8ffa72a thru 0x90081b9 (0xda90 bytes) 2024-09-13T07:12:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:486 write 0x39b1136 thru 0x39be896 (0xd761 bytes) 2024-09-13T07:12:38.516 INFO:teuthology.orchestra.run.smithi031.stdout:490 trunc from 0xe44731b to 0x9968c04 2024-09-13T07:12:38.530 INFO:teuthology.orchestra.run.smithi031.stdout:491 writesame 0x66d4e1d thru 0x66e047b data_size 0x1(0xb65f bytes) 2024-09-13T07:12:38.531 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:38.532 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:38.541 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:38.553 INFO:teuthology.orchestra.run.smithi031.stdout:492 punch from 0x69b0eda to 0x69b4f17, (0x403d bytes) 2024-09-13T07:12:38.553 INFO:teuthology.orchestra.run.smithi031.stdout:494 read 0x3ca7351 thru 0x3cad1ac (0x5e5c bytes) 2024-09-13T07:12:38.553 INFO:teuthology.orchestra.run.smithi031.stdout:495 write 0x399eef2 thru 0x39a65b8 (0x76c7 bytes) 2024-09-13T07:12:38.565 INFO:teuthology.orchestra.run.smithi031.stdout:496 writesame 0xd4263aa thru 0xd428003 data_size 0x13(0x1c5a bytes) 2024-09-13T07:12:38.565 INFO:teuthology.orchestra.run.smithi031.stdout:497 writesame 0x5ac87f0 thru 0x5acdad3 data_size 0xa(0x52e4 bytes) 2024-09-13T07:12:38.582 INFO:teuthology.orchestra.run.smithi031.stdout:499 trunc from 0xd428004 to 0x12fe49c 2024-09-13T07:12:38.582 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:38.583+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:38.603 INFO:teuthology.orchestra.run.smithi031.stdout:500 writesame 0x511ea6 thru 0x51948b data_size 0x1(0x75e6 bytes) 2024-09-13T07:12:38.641 INFO:teuthology.orchestra.run.smithi031.stdout:501 write 0xd8eae16 thru 0xd8f7160 (0xc34b bytes) 2024-09-13T07:12:38.646 INFO:teuthology.orchestra.run.smithi031.stdout:503 write 0x1f90a97 thru 0x1f9ab43 (0xa0ad bytes) 2024-09-13T07:12:38.650 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.650 INFO:teuthology.orchestra.run.smithi031.stdout:506 writesame 0x375b6e thru 0x37ab03 data_size 0x3d(0x4f96 bytes) 2024-09-13T07:12:38.652 INFO:teuthology.orchestra.run.smithi031.stdout:507 write 0x5929ca7 thru 0x593409c (0xa3f6 bytes) 2024-09-13T07:12:38.657 INFO:teuthology.orchestra.run.smithi031.stdout:508 punch from 0x66c992f to 0x66d43d9, (0xaaaa bytes) 2024-09-13T07:12:38.657 INFO:teuthology.orchestra.run.smithi031.stdout:509 read 0x7ce2396 thru 0x7cea7a6 (0x8411 bytes) 2024-09-13T07:12:38.663 INFO:teuthology.orchestra.run.smithi031.stdout:512 trunc from 0xd8f7161 to 0x7869a94 2024-09-13T07:12:38.674 INFO:teuthology.orchestra.run.smithi031.stdout:513 punch from 0x757e3d7 to 0x758bc99, (0xd8c2 bytes) 2024-09-13T07:12:38.674 INFO:teuthology.orchestra.run.smithi031.stdout:514 write 0x18a543b thru 0x18b330c (0xded2 bytes) 2024-09-13T07:12:38.683 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:38.684+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:38.684 INFO:teuthology.orchestra.run.smithi031.stdout:515 trunc from 0x7869a94 to 0x1ab501b 2024-09-13T07:12:38.692 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:38.692 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:38.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:38.710 INFO:teuthology.orchestra.run.smithi031.stdout:516 write 0x7bd7dc9 thru 0x7be4d23 (0xcf5b bytes) 2024-09-13T07:12:38.713 INFO:teuthology.orchestra.run.smithi031.stdout:517 writesame 0x3f172e4 thru 0x3f227c0 data_size 0x1(0xb4dd bytes) 2024-09-13T07:12:38.737 INFO:teuthology.orchestra.run.smithi031.stdout:518 punch from 0x7ac08b6 to 0x7ac8feb, (0x8735 bytes) 2024-09-13T07:12:38.737 INFO:teuthology.orchestra.run.smithi031.stdout:520 read 0x6b73745 thru 0x6b82dc7 (0xf683 bytes) 2024-09-13T07:12:38.737 INFO:teuthology.orchestra.run.smithi031.stdout:521 read 0x310f103 thru 0x311cc46 (0xdb44 bytes) 2024-09-13T07:12:38.737 INFO:teuthology.orchestra.run.smithi031.stdout:522 punch from 0x476b102 to 0x477ac0a, (0xfb08 bytes) 2024-09-13T07:12:38.737 INFO:teuthology.orchestra.run.smithi031.stdout:524 flatten 2024-09-13T07:12:38.751 INFO:teuthology.orchestra.run.smithi031.stdout:525 write 0xa76ae1c thru 0xa7772d2 (0xc4b7 bytes) 2024-09-13T07:12:38.754 INFO:teuthology.orchestra.run.smithi031.stdout:527 write 0x7de2e51 thru 0x7df03c7 (0xd577 bytes) 2024-09-13T07:12:38.766 INFO:teuthology.orchestra.run.smithi031.stdout:528 write 0xc83f523 thru 0xc8441f7 (0x4cd5 bytes) 2024-09-13T07:12:38.784 INFO:teuthology.orchestra.run.smithi031.stdout:529 trunc from 0xc8441f8 to 0x1237380 2024-09-13T07:12:38.785 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:38.785+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:38.807 INFO:teuthology.orchestra.run.smithi031.stdout:532 write 0x524a582 thru 0x524b4f2 (0xf71 bytes) 2024-09-13T07:12:38.809 INFO:teuthology.orchestra.run.smithi031.stdout:533 punch from 0x185a696 to 0x1864d24, (0xa68e bytes) 2024-09-13T07:12:38.809 INFO:teuthology.orchestra.run.smithi031.stdout:534 read 0x2c859a3 thru 0x2c94a79 (0xf0d7 bytes) 2024-09-13T07:12:38.809 INFO:teuthology.orchestra.run.smithi031.stdout:536 read 0x4ab7387 thru 0x4ac52ab (0xdf25 bytes) 2024-09-13T07:12:38.811 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:38.811 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:38.812 INFO:teuthology.orchestra.run.smithi031.stdout:537 trunc from 0x524b4f3 to 0x7df8208 2024-09-13T07:12:38.818 INFO:teuthology.orchestra.run.smithi031.stdout:538 writesame 0x5ffa2ba thru 0x5ffe44e data_size 0x1(0x4195 bytes) 2024-09-13T07:12:38.821 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:12:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:540 read 0x26623f4 thru 0x266b1d1 (0x8dde bytes) 2024-09-13T07:12:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:541 trunc from 0x7df8208 to 0xb6d8cf4 2024-09-13T07:12:38.837 INFO:teuthology.orchestra.run.smithi031.stdout:542 punch from 0x6b8aa99 to 0x6b8e55c, (0x3ac3 bytes) 2024-09-13T07:12:38.837 INFO:teuthology.orchestra.run.smithi031.stdout:543 write 0x3417014 thru 0x3420eca (0x9eb7 bytes) 2024-09-13T07:12:38.840 INFO:teuthology.orchestra.run.smithi031.stdout:545 writesame 0x7fe9115 thru 0x7ff414d data_size 0x1(0xb039 bytes) 2024-09-13T07:12:38.864 INFO:teuthology.orchestra.run.smithi031.stdout:546 write 0xa446af7 thru 0xa454ae9 (0xdff3 bytes) 2024-09-13T07:12:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:548 writesame 0x5b01e0b thru 0x5b10d47 data_size 0xf(0xef3d bytes) 2024-09-13T07:12:38.871 INFO:teuthology.orchestra.run.smithi031.stdout:549 read 0xb40c00f thru 0xb416eef (0xaee1 bytes) 2024-09-13T07:12:38.871 INFO:teuthology.orchestra.run.smithi031.stdout:550 write 0xd4a30f thru 0xd57b74 (0xd866 bytes) 2024-09-13T07:12:38.883 INFO:teuthology.orchestra.run.smithi031.stdout:551 trunc from 0xb6d8cf4 to 0x371fac0 2024-09-13T07:12:38.886 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:38.886+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:38.910 INFO:teuthology.orchestra.run.smithi031.stdout:553 write 0xb2a7678 thru 0xb2b329b (0xbc24 bytes) 2024-09-13T07:12:38.923 INFO:teuthology.orchestra.run.smithi031.stdout:554 write 0xe765801 thru 0xe76e1e0 (0x89e0 bytes) 2024-09-13T07:12:38.926 INFO:teuthology.orchestra.run.smithi031.stdout:555 writesame 0x803b32c thru 0x8047a45 data_size 0xf(0xc71a bytes) 2024-09-13T07:12:38.930 INFO:teuthology.orchestra.run.smithi031.stdout:556 write 0x964e231 thru 0x965b94a (0xd71a bytes) 2024-09-13T07:12:38.948 INFO:teuthology.orchestra.run.smithi031.stdout:557 trunc from 0xe76e1e1 to 0xf2ac20 2024-09-13T07:12:38.973 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:38.973 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:38.982 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:12:38.988 INFO:teuthology.orchestra.run.smithi031.stdout:559 write 0x811e6cb thru 0x812c813 (0xe149 bytes) 2024-09-13T07:12:38.991 INFO:teuthology.orchestra.run.smithi031.stdout:560 read 0x4d02ce8 thru 0x4d0f96c (0xcc85 bytes) 2024-09-13T07:12:38.991 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.991 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:38.994 INFO:teuthology.orchestra.run.smithi031.stdout:563 trunc from 0x812c814 to 0xad088e8 2024-09-13T07:12:39.001 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.002 INFO:teuthology.orchestra.run.smithi031.stdout:565 punch from 0x3915003 to 0x3923117, (0xe114 bytes) 2024-09-13T07:12:39.002 INFO:teuthology.orchestra.run.smithi031.stdout:566 writesame 0x4493fcd thru 0x449dde0 data_size 0x43(0x9e14 bytes) 2024-09-13T07:12:39.005 INFO:teuthology.orchestra.run.smithi031.stdout:567 punch from 0x5723f17 to 0x5724327, (0x410 bytes) 2024-09-13T07:12:39.011 INFO:teuthology.orchestra.run.smithi031.stdout:568 trunc from 0xad088e8 to 0x5208072 2024-09-13T07:12:39.026 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:39.027+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:39.027 INFO:teuthology.orchestra.run.smithi031.stdout:569 trunc from 0x5208072 to 0x8e5aca3 2024-09-13T07:12:39.034 INFO:teuthology.orchestra.run.smithi031.stdout:570 writesame 0x37c34ff thru 0x37caec3 data_size 0x1(0x79c5 bytes) 2024-09-13T07:12:39.050 INFO:teuthology.orchestra.run.smithi031.stdout:571 read 0x625dde7 thru 0x626aa76 (0xcc90 bytes) 2024-09-13T07:12:39.050 INFO:teuthology.orchestra.run.smithi031.stdout:573 read 0x7bd3a04 thru 0x7bdc816 (0x8e13 bytes) 2024-09-13T07:12:39.050 INFO:teuthology.orchestra.run.smithi031.stdout:577 read 0x40f03a1 thru 0x40fe15a (0xddba bytes) 2024-09-13T07:12:39.051 INFO:teuthology.orchestra.run.smithi031.stdout:579 punch from 0x41a1409 to 0x41aaa55, (0x964c bytes) 2024-09-13T07:12:39.051 INFO:teuthology.orchestra.run.smithi031.stdout:580 punch from 0x7b50380 to 0x7b5a2d5, (0x9f55 bytes) 2024-09-13T07:12:39.051 INFO:teuthology.orchestra.run.smithi031.stdout:583 read 0x236a224 thru 0x2373a44 (0x9821 bytes) 2024-09-13T07:12:39.064 INFO:teuthology.orchestra.run.smithi031.stdout:584 write 0xdc4bbe0 thru 0xdc5bb85 (0xffa6 bytes) 2024-09-13T07:12:39.069 INFO:teuthology.orchestra.run.smithi031.stdout:586 punch from 0x1ea4bff to 0x1ea637d, (0x177e bytes) 2024-09-13T07:12:39.069 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.069 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.069 INFO:teuthology.orchestra.run.smithi031.stdout:590 write 0x50b725a thru 0x50b83fd (0x11a4 bytes) 2024-09-13T07:12:39.074 INFO:teuthology.orchestra.run.smithi031.stdout:591 read 0x25385b0 thru 0x253b2cd (0x2d1e bytes) 2024-09-13T07:12:39.074 INFO:teuthology.orchestra.run.smithi031.stdout:592 write 0x53e0567 thru 0x53e8f43 (0x89dd bytes) 2024-09-13T07:12:39.078 INFO:teuthology.orchestra.run.smithi031.stdout:593 read 0x34a7407 thru 0x34b3f98 (0xcb92 bytes) 2024-09-13T07:12:39.078 INFO:teuthology.orchestra.run.smithi031.stdout:594 read 0x1915ceb thru 0x19239ea (0xdd00 bytes) 2024-09-13T07:12:39.079 INFO:teuthology.orchestra.run.smithi031.stdout:595 write 0x960e494 thru 0x9618ae6 (0xa653 bytes) 2024-09-13T07:12:39.083 INFO:teuthology.orchestra.run.smithi031.stdout:596 writesame 0x48f440f thru 0x4901bac data_size 0x8f(0xd79e bytes) 2024-09-13T07:12:39.087 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:39.088+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:39.088 INFO:teuthology.orchestra.run.smithi031.stdout:597 write 0x47de5ff thru 0x47e24a8 (0x3eaa bytes) 2024-09-13T07:12:39.091 INFO:teuthology.orchestra.run.smithi031.stdout:599 read 0xbaa8174 thru 0xbab666a (0xe4f7 bytes) 2024-09-13T07:12:39.092 INFO:teuthology.orchestra.run.smithi031.stdout:600 punch from 0xbb88d1e to 0xbb89ba9, (0xe8b bytes) 2024-09-13T07:12:39.092 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.092 INFO:teuthology.orchestra.run.smithi031.stdout:602 punch from 0x5d3951a to 0x5d3ad28, (0x180e bytes) 2024-09-13T07:12:39.092 INFO:teuthology.orchestra.run.smithi031.stdout:604 write 0x63cc4b5 thru 0x63cdea8 (0x19f4 bytes) 2024-09-13T07:12:39.092 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:39.092 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:39.095 INFO:teuthology.orchestra.run.smithi031.stdout:606 write 0x620b778 thru 0x621977c (0xe005 bytes) 2024-09-13T07:12:39.101 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:12:39.103 INFO:teuthology.orchestra.run.smithi031.stdout:607 trunc from 0xdc5bb86 to 0xa584aa6 2024-09-13T07:12:39.118 INFO:teuthology.orchestra.run.smithi031.stdout:609 trunc from 0xa584aa6 to 0xdd171d4 2024-09-13T07:12:39.123 INFO:teuthology.orchestra.run.smithi031.stdout:611 writesame 0x2dd71b3 thru 0x2dd976a data_size 0x2(0x25b8 bytes) 2024-09-13T07:12:39.127 INFO:teuthology.orchestra.run.smithi031.stdout:612 punch from 0x55eef30 to 0x55f1a8f, (0x2b5f bytes) 2024-09-13T07:12:39.127 INFO:teuthology.orchestra.run.smithi031.stdout:613 punch from 0x7816930 to 0x7824290, (0xd960 bytes) 2024-09-13T07:12:39.140 INFO:teuthology.orchestra.run.smithi031.stdout:614 trunc from 0xdd171d4 to 0x29e5a88 2024-09-13T07:12:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:615 write 0xeb03f54 thru 0xeb119b1 (0xda5e bytes) 2024-09-13T07:12:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:619 write 0xaa73a9d thru 0xaa7952a (0x5a8e bytes) 2024-09-13T07:12:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:621 read 0xc13587c thru 0xc13b930 (0x60b5 bytes) 2024-09-13T07:12:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:622 writesame 0x39a630 thru 0x39c154 data_size 0x1(0x1b25 bytes) 2024-09-13T07:12:39.181 INFO:teuthology.orchestra.run.smithi031.stdout:623 read 0x8b3ebab thru 0x8b4327c (0x46d2 bytes) 2024-09-13T07:12:39.188 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:39.189+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:39.188 INFO:teuthology.orchestra.run.smithi031.stdout:625 trunc from 0xeb119b2 to 0x8ab06dc 2024-09-13T07:12:39.201 INFO:teuthology.orchestra.run.smithi031.stdout:628 read 0x4a0632b thru 0x4a0b893 (0x5569 bytes) 2024-09-13T07:12:39.201 INFO:teuthology.orchestra.run.smithi031.stdout:629 writesame 0x3019072 thru 0x301f1df data_size 0x3(0x616e bytes) 2024-09-13T07:12:39.205 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.205 INFO:teuthology.orchestra.run.smithi031.stdout:632 read 0x531b1c0 thru 0x531cb95 (0x19d6 bytes) 2024-09-13T07:12:39.205 INFO:teuthology.orchestra.run.smithi031.stdout:633 writesame 0x4248759 thru 0x42516c5 data_size 0x1(0x8f6d bytes) 2024-09-13T07:12:39.223 INFO:teuthology.orchestra.run.smithi031.stdout:634 write 0x6434e74 thru 0x643be03 (0x6f90 bytes) 2024-09-13T07:12:39.227 INFO:teuthology.orchestra.run.smithi031.stdout:635 read 0x676afd1 thru 0x67743a2 (0x93d2 bytes) 2024-09-13T07:12:39.227 INFO:teuthology.orchestra.run.smithi031.stdout:636 punch from 0x6b81177 to 0x6b8f72e, (0xe5b7 bytes) 2024-09-13T07:12:39.227 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.227 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:39.227 INFO:teuthology.orchestra.run.smithi031.stdout:639 punch from 0x1f81834 to 0x1f90150, (0xe91c bytes) 2024-09-13T07:12:39.227 INFO:teuthology.orchestra.run.smithi031.stdout:641 clone 4 order 20 su 262144 sc 3 2024-09-13T07:12:39.251 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:39.251 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:39.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:39.304 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:39.305+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:39.367 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:39.368 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:39.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:39.390 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:39.391+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:39.490 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:39.491+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:39.523 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:39.523 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:39.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:39.591 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:39.592+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:39.616 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:12:39.616 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:12:39.616 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T07:12:39.616 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.855318 2024-09-13T07:12:39.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.8553180958829181 2024-09-13T07:12:39.638 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:39.638 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:39.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:39.731 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:39.732+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:39.793 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:39.794+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:39.799 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:39.799 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:39.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:39.894 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:39.895+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:39.955 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:39.955 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:39.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:39.995 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:39.996+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:40.095 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:40.096+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:40.110 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:40.110 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:40.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:40.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:12:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 0.855318 (8560542) 2024-09-13T07:12:40.234 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:40.235+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:40.274 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:40.275 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:40.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:40.327 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:40.328+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:40.339 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T07:12:40.341 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.341 INFO:teuthology.orchestra.run.smithi031.stdout:643 write 0x1f7f652 thru 0x1f829cc (0x337b bytes) 2024-09-13T07:12:40.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:12:40.353 INFO:teuthology.orchestra.run.smithi031.stdout:644 read 0x708ab59 thru 0x70928b4 (0x7d5c bytes) 2024-09-13T07:12:40.354 INFO:teuthology.orchestra.run.smithi031.stdout:645 read 0x34de85b thru 0x34e6f50 (0x86f6 bytes) 2024-09-13T07:12:40.365 INFO:teuthology.orchestra.run.smithi031.stdout:646 writesame 0xbb91040 thru 0xbb9128d data_size 0x5(0x24e bytes) 2024-09-13T07:12:40.365 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.365 INFO:teuthology.orchestra.run.smithi031.stdout:648 writesame 0xb2ae9e3 thru 0xb2aeb67 data_size 0x1(0x185 bytes) 2024-09-13T07:12:40.369 INFO:teuthology.orchestra.run.smithi031.stdout:649 punch from 0x9a6c39f to 0x9a766eb, (0xa34c bytes) 2024-09-13T07:12:40.369 INFO:teuthology.orchestra.run.smithi031.stdout:651 read 0x3cb1962 thru 0x3cbd2f5 (0xb994 bytes) 2024-09-13T07:12:40.369 INFO:teuthology.orchestra.run.smithi031.stdout:652 write 0x65b03e0 thru 0x65b51c1 (0x4de2 bytes) 2024-09-13T07:12:40.373 INFO:teuthology.orchestra.run.smithi031.stdout:653 punch from 0x2847e37 to 0x284f1ab, (0x7374 bytes) 2024-09-13T07:12:40.385 INFO:teuthology.orchestra.run.smithi031.stdout:654 writesame 0xeb1291c thru 0xeb1f79d data_size 0x63(0xce82 bytes) 2024-09-13T07:12:40.385 INFO:teuthology.orchestra.run.smithi031.stdout:655 write 0x6bbcccc thru 0x6bbf9b6 (0x2ceb bytes) 2024-09-13T07:12:40.389 INFO:teuthology.orchestra.run.smithi031.stdout:657 writesame 0x5e249b1 thru 0x5e323a8 data_size 0xc8(0xd9f8 bytes) 2024-09-13T07:12:40.392 INFO:teuthology.orchestra.run.smithi031.stdout:661 write 0x434fa65 thru 0x4354370 (0x490c bytes) 2024-09-13T07:12:40.397 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:40.397+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:40.399 INFO:teuthology.orchestra.run.smithi031.stdout:662 write 0x5fc378e thru 0x5fd2460 (0xecd3 bytes) 2024-09-13T07:12:40.403 INFO:teuthology.orchestra.run.smithi031.stdout:664 read 0xa7f8452 thru 0xa7fd602 (0x51b1 bytes) 2024-09-13T07:12:40.403 INFO:teuthology.orchestra.run.smithi031.stdout:665 write 0x650e980 thru 0x65157c6 (0x6e47 bytes) 2024-09-13T07:12:40.407 INFO:teuthology.orchestra.run.smithi031.stdout:667 punch from 0x2aa8769 to 0x2ab01ab, (0x7a42 bytes) 2024-09-13T07:12:40.408 INFO:teuthology.orchestra.run.smithi031.stdout:668 read 0x7ec7ee4 thru 0x7ecc42d (0x454a bytes) 2024-09-13T07:12:40.408 INFO:teuthology.orchestra.run.smithi031.stdout:669 read 0x73b2148 thru 0x73ba30a (0x81c3 bytes) 2024-09-13T07:12:40.408 INFO:teuthology.orchestra.run.smithi031.stdout:670 read 0x58243d2 thru 0x5831a58 (0xd687 bytes) 2024-09-13T07:12:40.408 INFO:teuthology.orchestra.run.smithi031.stdout:673 write 0xdcd7d72 thru 0xdcdafe4 (0x3273 bytes) 2024-09-13T07:12:40.412 INFO:teuthology.orchestra.run.smithi031.stdout:674 writesame 0x19763c7 thru 0x1981c77 data_size 0x1(0xb8b1 bytes) 2024-09-13T07:12:40.435 INFO:teuthology.orchestra.run.smithi031.stdout:675 read 0x6bd037c thru 0x6bddffb (0xdc80 bytes) 2024-09-13T07:12:40.435 INFO:teuthology.orchestra.run.smithi031.stdout:676 read 0x7025232 thru 0x7033cb7 (0xea86 bytes) 2024-09-13T07:12:40.435 INFO:teuthology.orchestra.run.smithi031.stdout:677 read 0x800b30b thru 0x80168fc (0xb5f2 bytes) 2024-09-13T07:12:40.435 INFO:teuthology.orchestra.run.smithi031.stdout:679 write 0x5339247 thru 0x534232c (0x90e6 bytes) 2024-09-13T07:12:40.442 INFO:teuthology.orchestra.run.smithi031.stdout:680 write 0x239a7de thru 0x23a7411 (0xcc34 bytes) 2024-09-13T07:12:40.446 INFO:teuthology.orchestra.run.smithi031.stdout:681 read 0xa86b95b thru 0xa86d833 (0x1ed9 bytes) 2024-09-13T07:12:40.458 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:40.458 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:40.464 INFO:teuthology.orchestra.run.smithi031.stdout:686 trunc from 0xeb1f79e to 0xb14755 2024-09-13T07:12:40.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:40.493 INFO:teuthology.orchestra.run.smithi031.stdout:687 write 0xcf3a59 thru 0xcf67d6 (0x2d7e bytes) 2024-09-13T07:12:40.496 INFO:teuthology.orchestra.run.smithi031.stdout:688 read 0x4dae52 thru 0x4e6a54 (0xbc03 bytes) 2024-09-13T07:12:40.497 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:40.498+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:40.499 INFO:teuthology.orchestra.run.smithi031.stdout:689 trunc from 0xcf67d7 to 0x2e4b361 2024-09-13T07:12:40.501 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:40.505 INFO:teuthology.orchestra.run.smithi031.stdout:691 read 0x2275fee thru 0x227ecfa (0x8d0d bytes) 2024-09-13T07:12:40.507 INFO:teuthology.orchestra.run.smithi031.stdout:692 trunc from 0x2e4b361 to 0xe91903 2024-09-13T07:12:40.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:40.522 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:40.525 INFO:teuthology.orchestra.run.smithi031.stdout:693 trunc from 0xe91903 to 0xb94a0a7 2024-09-13T07:12:40.531 INFO:teuthology.orchestra.run.smithi031.stdout:694 trunc from 0xb94a0a7 to 0xb6cd221 2024-09-13T07:12:40.541 INFO:teuthology.orchestra.run.smithi031.stdout:695 trunc from 0xb6cd221 to 0x74c6a16 2024-09-13T07:12:40.543 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:40.543 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:40.547 INFO:teuthology.orchestra.run.smithi031.stdout:697 writesame 0x3cf59a0 thru 0x3d017bd data_size 0x3e(0xbe1e bytes) 2024-09-13T07:12:40.554 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.554 INFO:teuthology.orchestra.run.smithi031.stdout:700 read 0x46d973c thru 0x46e0987 (0x724c bytes) 2024-09-13T07:12:40.559 INFO:teuthology.orchestra.run.smithi031.stdout:701 trunc from 0x74c6a16 to 0x30d6468 2024-09-13T07:12:40.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:40.571 INFO:teuthology.orchestra.run.smithi031.stdout:702 punch from 0x1443fef to 0x144b23e, (0x724f bytes) 2024-09-13T07:12:40.571 INFO:teuthology.orchestra.run.smithi031.stdout:706 read 0x3fb06e thru 0x404e97 (0x9e2a bytes) 2024-09-13T07:12:40.581 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:40.582 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:40.582 INFO:teuthology.orchestra.run.smithi031.stdout:707 write 0x69e2c00 thru 0x69ef5e3 (0xc9e4 bytes) 2024-09-13T07:12:40.585 INFO:teuthology.orchestra.run.smithi031.stdout:709 punch from 0x10644d0 to 0x106eff1, (0xab21 bytes) 2024-09-13T07:12:40.585 INFO:teuthology.orchestra.run.smithi031.stdout:710 read 0x5d16c32 thru 0x5d1bf2b (0x52fa bytes) 2024-09-13T07:12:40.585 INFO:teuthology.orchestra.run.smithi031.stdout:711 punch from 0x4dbb6dd to 0x4dbc713, (0x1036 bytes) 2024-09-13T07:12:40.585 INFO:teuthology.orchestra.run.smithi031.stdout:712 read 0x487f59e thru 0x4882bf8 (0x365b bytes) 2024-09-13T07:12:40.585 INFO:teuthology.orchestra.run.smithi031.stdout:715 write 0x5851c95 thru 0x5859092 (0x73fe bytes) 2024-09-13T07:12:40.588 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.588 INFO:teuthology.orchestra.run.smithi031.stdout:719 read 0xfa0d52 thru 0xfae5bc (0xd86b bytes) 2024-09-13T07:12:40.588 INFO:teuthology.orchestra.run.smithi031.stdout:721 read 0x1236bfc thru 0x123cfc5 (0x63ca bytes) 2024-09-13T07:12:40.588 INFO:teuthology.orchestra.run.smithi031.stdout:722 punch from 0x5f19180 to 0x5f255e7, (0xc467 bytes) 2024-09-13T07:12:40.588 INFO:teuthology.orchestra.run.smithi031.stdout:723 writesame 0x2ab7b8b thru 0x2abf24c data_size 0x1b(0x76c2 bytes) 2024-09-13T07:12:40.597 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:40.598+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:40.601 INFO:teuthology.orchestra.run.smithi031.stdout:724 write 0xa2c1d14 thru 0xa2c441f (0x270c bytes) 2024-09-13T07:12:40.604 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:40.607 INFO:teuthology.orchestra.run.smithi031.stdout:725 trunc from 0xa2c4420 to 0xd49516e 2024-09-13T07:12:40.613 INFO:teuthology.orchestra.run.smithi031.stdout:726 read 0x277df93 thru 0x278018a (0x21f8 bytes) 2024-09-13T07:12:40.613 INFO:teuthology.orchestra.run.smithi031.stdout:728 writesame 0x6c3bf8d thru 0x6c4913d data_size 0x1(0xd1b1 bytes) 2024-09-13T07:12:40.623 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:40.624 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:40.624 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:789 3.9 e24/19 3:9280c38f:::rbd_data.1173f77ecad8.000000000000010c:head v 24'3, mlcod=23'1)", 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:36.637141+0000", 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.9870048489999999, 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087117599999999998, 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637141+0000", 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637141+0000", 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637140+0000", 2024-09-13T07:12:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637241+0000", 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010092199999999999 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637257+0000", 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6523000000000001e-05 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637264+0000", 2024-09-13T07:12:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637308+0000", 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4521000000000003e-05 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637325+0000", 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7221999999999999e-05 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.637968+0000", 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064206599999999995 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:36.638012+0000", 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4589000000000003e-05 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 24 entries 25'5 (0'0) error 3:965bf927:::rbd_data.1173f77ecad8.0000000000000048:head by client.4467.0:1071 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 24'3)", 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.290796+0000", 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3333496220000001, 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020184669999999999, 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290796+0000", 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290796+0000", 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290796+0000", 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290803+0000", 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:12:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290828+0000", 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5241000000000001e-05 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290843+0000", 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4415e-05 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.291675+0000", 2024-09-13T07:12:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083257100000000005 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.292815+0000", 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001139529 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1081 3.9 e25/19 3:965bf927:::rbd_data.1173f77ecad8.0000000000000048:head v 25'6, mlcod=24'4)", 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.293578+0000", 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3305674339999998, 2024-09-13T07:12:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021262339999999998, 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293578+0000", 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293578+0000", 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293578+0000", 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293586+0000", 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5799999999999992e-06 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293597+0000", 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1245e-05 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293605+0000", 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5340000000000002e-06 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293645+0000", 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0292e-05 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293665+0000", 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9647999999999999e-05 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.295665+0000", 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020006339999999998 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.295705+0000", 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9131000000000002e-05 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.0 epoch 25/19 rep_tid 54 entries 25'5 (0'0) error 3:92f09a00:::rbd_data.1173f77ecad8.0000000000000052:head by client.4467.0:1089 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 22'3)", 2024-09-13T07:12:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297025+0000", 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.327120442, 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001701641, 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297025+0000", 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297025+0000", 2024-09-13T07:12:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297025+0000", 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297031+0000", 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.246e-06 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297050+0000", 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8349e-05 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297064+0000", 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3913e-05 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297126+0000", 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2860999999999993e-05 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298727+0000", 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001600567 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3262240599999999, 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:12:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.324746035, 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3219289860000001, 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1148 3.9 e25/19 3:933222f0:::rbd_data.1173f77ecad8.00000000000000e8:head v 25'10, mlcod=25'7)", 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.327126+0000", 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.2970195470000001, 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001480208, 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327126+0000", 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327126+0000", 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327125+0000", 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327151+0000", 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6313e-05 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327174+0000", 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2396e-05 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327183+0000", 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327241+0000", 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8335999999999998e-05 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327270+0000", 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9071e-05 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328571+0000", 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013009409999999999 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328606+0000", 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5577000000000002e-05 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1151 3.9 e25/19 3:9280c38f:::rbd_data.1173f77ecad8.000000000000010c:head v 25'12, mlcod=25'7)", 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.327962+0000", 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.2961836880000002, 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010669290000000001, 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327962+0000", 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327962+0000", 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327961+0000", 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327969+0000", 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9359999999999999e-06 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327981+0000", 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1814000000000001e-05 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327988+0000", 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.81e-06 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328028+0000", 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9888000000000002e-05 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328048+0000", 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0254999999999998e-05 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328996+0000", 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094780600000000004 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.329029+0000", 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3231000000000001e-05 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1285 3.4 e25/19 3:2712c73b:::rbd_data.1173c53667c.000000000000008e:head v 25'4, mlcod=23'2)", 2024-09-13T07:12:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.441598+0000", 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.182547612, 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00098853900000000008, 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441598+0000", 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441598+0000", 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441597+0000", 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441643+0000", 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5939999999999997e-05 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441658+0000", 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5659999999999999e-05 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441665+0000", 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441712+0000", 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7002000000000002e-05 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.441738+0000", 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5987999999999999e-05 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.442562+0000", 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082434800000000003 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.442587+0000", 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4692999999999999e-05 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1319 3.0 e25/19 3:06065967:::rbd_data.1173c53667c.0000000000000056:head v 25'7, mlcod=25'5)", 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1512435910000001, 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026907649999999999, 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472901+0000", 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472909+0000", 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472925+0000", 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6019e-05 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472933+0000", 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8089999999999993e-06 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472979+0000", 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6034e-05 2024-09-13T07:12:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.473001+0000", 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2218999999999999e-05 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475552+0000", 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025503309999999999 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475593+0000", 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1468999999999999e-05 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1347 3.0 e25/19 3:06065967:::rbd_data.1173c53667c.0000000000000056:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.517150+0000", 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.106995875, 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087794200000000005, 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517150+0000", 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517150+0000", 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517148+0000", 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517174+0000", 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5508999999999999e-05 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517194+0000", 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0647000000000001e-05 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517204+0000", 2024-09-13T07:12:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517252+0000", 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8007999999999998e-05 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517272+0000", 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9983000000000002e-05 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.517990+0000", 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071786099999999998 2024-09-13T07:12:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.518028+0000", 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7675000000000001e-05 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1594 3.9 e25/19 3:929e4997:::rbd_data.1173c53667c.0000000000000005:head v 25'13, mlcod=25'10)", 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.813122+0000", 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.8110242960000003, 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011636789999999999, 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813122+0000", 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813122+0000", 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813120+0000", 2024-09-13T07:12:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813217+0000", 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6700000000000006e-05 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813232+0000", 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5298000000000002e-05 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813238+0000", 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1279999999999996e-06 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813294+0000", 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6010000000000001e-05 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.813320+0000", 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5633e-05 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.814257+0000", 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093668500000000003 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.814285+0000", 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8558e-05 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1665 3.4 e25/19 3:21741da7:::rbd_data.1173c53667c.000000000000001e:head v 25'10, mlcod=25'8)", 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.945790+0000", 2024-09-13T07:12:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.6783558149999998, 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001160906, 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945790+0000", 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945790+0000", 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945788+0000", 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945799+0000", 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1123e-05 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945821+0000", 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2048e-05 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945831+0000", 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.006e-05 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945895+0000", 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3866000000000001e-05 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.945922+0000", 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6177999999999999e-05 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946912+0000", 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099086399999999994 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946951+0000", 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000001e-05 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1666 3.4 e25/19 3:206d0453:::rbd_data.1173c53667c.000000000000001f:head v 25'11, mlcod=25'8)", 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.946186+0000", 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.677959763, 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00146028, 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946186+0000", 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946186+0000", 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946185+0000", 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946200+0000", 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4739000000000001e-05 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946218+0000", 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7547999999999999e-05 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946223+0000", 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8640000000000001e-06 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946261+0000", 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7638000000000001e-05 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.946281+0000", 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9536000000000001e-05 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.947613+0000", 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013322620000000001 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.947646+0000", 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3461999999999999e-05 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:2802 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'50, mlcod=27'48)", 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.549821+0000", 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.074324873999999999, 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091378200000000003, 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549821+0000", 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549821+0000", 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549820+0000", 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549829+0000", 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8820000000000007e-06 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549844+0000", 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5630000000000001e-05 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549852+0000", 2024-09-13T07:12:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7910000000000005e-06 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549903+0000", 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1218999999999999e-05 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.549926+0000", 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2847999999999999e-05 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.550704+0000", 2024-09-13T07:12:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077820400000000001 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.550735+0000", 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0372999999999999e-05 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:2806 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'52, mlcod=27'50)", 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.561394+0000", 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.062751591999999995, 2024-09-13T07:12:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00090185799999999996, 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561394+0000", 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561394+0000", 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561393+0000", 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561404+0000", 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0696000000000001e-05 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561424+0000", 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0513000000000002e-05 2024-09-13T07:12:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561433+0000", 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561498+0000", 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4930000000000003e-05 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.561524+0000", 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6207000000000002e-05 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.562264+0000", 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000739877 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.562296+0000", 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1754000000000001e-05 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:2821 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'55, mlcod=27'53)", 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.578295+0000", 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045850782999999999, 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001343856, 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578295+0000", 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578295+0000", 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578294+0000", 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578305+0000", 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0509000000000001e-05 2024-09-13T07:12:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578316+0000", 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0912e-05 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578325+0000", 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9809999999999992e-06 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578385+0000", 2024-09-13T07:12:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9945000000000003e-05 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.578410+0000", 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5766e-05 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.579597+0000", 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011866159999999999 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.579639+0000", 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980999999999997e-05 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:2832 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'57, mlcod=27'55)", 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.587712+0000", 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.036434274000000003, 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087327200000000002, 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587712+0000", 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587712+0000", 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587711+0000", 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587720+0000", 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1530000000000003e-06 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587734+0000", 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3797e-05 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587741+0000", 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4200000000000001e-06 2024-09-13T07:12:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587790+0000", 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9082000000000002e-05 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.587813+0000", 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3357999999999999e-05 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.588561+0000", 2024-09-13T07:12:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074721 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.588585+0000", 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4159999999999999e-05 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:2856 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'62, mlcod=27'60)", 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.615763+0000", 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083827369999999995, 2024-09-13T07:12:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010709459999999999, 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615763+0000", 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615763+0000", 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615762+0000", 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615770+0000", 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4839999999999998e-06 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615780+0000", 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0496000000000001e-05 2024-09-13T07:12:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615787+0000", 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1239999999999999e-06 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615842+0000", 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700999999999997e-05 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.615865+0000", 2024-09-13T07:12:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.353e-05 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.616803+0000", 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093769800000000005 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.616834+0000", 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0877000000000003e-05 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:40.709 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:40.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:40.711 INFO:teuthology.orchestra.run.smithi031.stdout:730 read 0xa2387 thru 0xa3935 (0x15af bytes) 2024-09-13T07:12:40.711 INFO:teuthology.orchestra.run.smithi031.stdout:733 read 0x33afcc7 thru 0x33ba955 (0xac8f bytes) 2024-09-13T07:12:40.711 INFO:teuthology.orchestra.run.smithi031.stdout:735 read 0x42f8038 thru 0x42f99ff (0x19c8 bytes) 2024-09-13T07:12:40.711 INFO:teuthology.orchestra.run.smithi031.stdout:736 read 0x1c4af23 thru 0x1c57724 (0xc802 bytes) 2024-09-13T07:12:40.711 INFO:teuthology.orchestra.run.smithi031.stdout:738 write 0xbc99f9 thru 0xbd5a52 (0xc05a bytes) 2024-09-13T07:12:40.712 INFO:teuthology.orchestra.run.smithi031.stdout:739 read 0x5de61bd thru 0x5def15c (0x8fa0 bytes) 2024-09-13T07:12:40.712 INFO:teuthology.orchestra.run.smithi031.stdout:740 write 0x250273a thru 0x250ba77 (0x933e bytes) 2024-09-13T07:12:40.712 INFO:teuthology.orchestra.run.smithi031.stdout:744 writesame 0x25e20bf thru 0x25f12ae data_size 0x9e(0xf1f0 bytes) 2024-09-13T07:12:40.712 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:12:40.712 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:12:40.713 INFO:teuthology.orchestra.run.smithi031.stdout:745 read 0xdec89e thru 0xdf8b4c (0xc2af bytes) 2024-09-13T07:12:40.713 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.713 INFO:teuthology.orchestra.run.smithi031.stdout:747 trunc from 0xd49516e to 0xcca9ee0 2024-09-13T07:12:40.713 INFO:teuthology.orchestra.run.smithi031.stdout:748 read 0xc244913 thru 0xc24fd22 (0xb410 bytes) 2024-09-13T07:12:40.714 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.714 INFO:teuthology.orchestra.run.smithi031.stdout:751 read 0x3ee01d6 thru 0x3ee82cc (0x80f7 bytes) 2024-09-13T07:12:40.714 INFO:teuthology.orchestra.run.smithi031.stdout:753 write 0xe3cbf1 thru 0xe41785 (0x4b95 bytes) 2024-09-13T07:12:40.714 INFO:teuthology.orchestra.run.smithi031.stdout:754 read 0x833b0b3 thru 0x834686b (0xb7b9 bytes) 2024-09-13T07:12:40.714 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.714 INFO:teuthology.orchestra.run.smithi031.stdout:757 write 0xf6c8ac thru 0xf755d3 (0x8d28 bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:760 punch from 0xd586eb to 0xd5cdd1, (0x46e6 bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:762 write 0x583af5 thru 0x58ff0c (0xc418 bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:763 read 0xaa8bf51 thru 0xaa8c79f (0x84f bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:764 write 0xd6f8a2e thru 0xd6fa60a (0x1bdd bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:766 trunc from 0xd6fa60b to 0x86e2e4a 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:768 write 0x83b31e6 thru 0x83b78b7 (0x46d2 bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:769 read 0x23ccab0 thru 0x23dbe0c (0xf35d bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:770 read 0x12aacf9 thru 0x12b11ee (0x64f6 bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:772 write 0x5f4bcb thru 0x5f9ea6 (0x52dc bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:773 writesame 0x38d9c6 thru 0x393e2a data_size 0xd(0x6465 bytes) 2024-09-13T07:12:40.716 INFO:teuthology.orchestra.run.smithi031.stdout:776 trunc from 0x86e2e4a to 0x1fa6f0d 2024-09-13T07:12:40.717 INFO:teuthology.orchestra.run.smithi031.stdout:777 read 0x1a9a2c0 thru 0x1aa010e (0x5e4f bytes) 2024-09-13T07:12:40.717 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:40.711+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:40.725 INFO:teuthology.orchestra.run.smithi031.stdout:778 trunc from 0x1fa6f0d to 0xcea3110 2024-09-13T07:12:40.732 INFO:teuthology.orchestra.run.smithi031.stdout:779 writesame 0x97fbc8e thru 0x9809d6a data_size 0x1d(0xe0dd bytes) 2024-09-13T07:12:40.737 INFO:teuthology.orchestra.run.smithi031.stdout:780 read 0x9c9a090 thru 0x9ca32c4 (0x9235 bytes) 2024-09-13T07:12:40.744 INFO:teuthology.orchestra.run.smithi031.stdout:781 write 0xdd9dc95 thru 0xdda1475 (0x37e1 bytes) 2024-09-13T07:12:40.761 INFO:teuthology.orchestra.run.smithi031.stdout:782 trunc from 0xdda1476 to 0x198334d 2024-09-13T07:12:40.775 INFO:teuthology.orchestra.run.smithi031.stdout:783 read 0x12fd7d thru 0x13aa50 (0xacd4 bytes) 2024-09-13T07:12:40.776 INFO:teuthology.orchestra.run.smithi031.stdout:784 read 0xc1918c thru 0xc1ca21 (0x3896 bytes) 2024-09-13T07:12:40.776 INFO:teuthology.orchestra.run.smithi031.stdout:785 punch from 0x9692fe to 0x9789d0, (0xf6d2 bytes) 2024-09-13T07:12:40.777 INFO:teuthology.orchestra.run.smithi031.stdout:786 read 0x191b06e thru 0x191ea88 (0x3a1b bytes) 2024-09-13T07:12:40.777 INFO:teuthology.orchestra.run.smithi031.stdout:788 read 0xf5ba25 thru 0xf5de46 (0x2422 bytes) 2024-09-13T07:12:40.777 INFO:teuthology.orchestra.run.smithi031.stdout:793 read 0xcd2f59 thru 0xcd4837 (0x18df bytes) 2024-09-13T07:12:40.777 INFO:teuthology.orchestra.run.smithi031.stdout:794 punch from 0x8cca94 to 0x8dc40e, (0xf97a bytes) 2024-09-13T07:12:40.789 INFO:teuthology.orchestra.run.smithi031.stdout:796 write 0x5c8efff thru 0x5c98146 (0x9148 bytes) 2024-09-13T07:12:40.800 INFO:teuthology.orchestra.run.smithi031.stdout:798 write 0x81b2a8e thru 0x81b9044 (0x65b7 bytes) 2024-09-13T07:12:40.804 INFO:teuthology.orchestra.run.smithi031.stdout:799 read 0xa2d03e thru 0xa366c0 (0x9683 bytes) 2024-09-13T07:12:40.804 INFO:teuthology.orchestra.run.smithi031.stdout:800 read 0xb7abb9 thru 0xb865fd (0xba45 bytes) 2024-09-13T07:12:40.809 INFO:teuthology.orchestra.run.smithi031.stdout:801 trunc from 0x81b9045 to 0x42450d1 2024-09-13T07:12:40.810 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:40.811+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:40.830 INFO:teuthology.orchestra.run.smithi031.stdout:802 write 0x7cbf228 thru 0x7cc56a1 (0x647a bytes) 2024-09-13T07:12:40.834 INFO:teuthology.orchestra.run.smithi031.stdout:803 punch from 0x4d8fb17 to 0x4d9f171, (0xf65a bytes) 2024-09-13T07:12:40.834 INFO:teuthology.orchestra.run.smithi031.stdout:804 write 0x11a769e thru 0x11b03d6 (0x8d39 bytes) 2024-09-13T07:12:40.839 INFO:teuthology.orchestra.run.smithi031.stdout:805 trunc from 0x7cc56a2 to 0xb02b9d6 2024-09-13T07:12:40.845 INFO:teuthology.orchestra.run.smithi031.stdout:806 write 0xf57301 thru 0xf5f7b7 (0x84b7 bytes) 2024-09-13T07:12:40.847 INFO:teuthology.orchestra.run.smithi031.stdout:807 punch from 0xa0c5dac to 0xa0c64a5, (0x6f9 bytes) 2024-09-13T07:12:40.847 INFO:teuthology.orchestra.run.smithi031.stdout:811 read 0xac4b501 thru 0xac4c9fe (0x14fe bytes) 2024-09-13T07:12:40.847 INFO:teuthology.orchestra.run.smithi031.stdout:814 write 0xaa6b0dd thru 0xaa732a4 (0x81c8 bytes) 2024-09-13T07:12:40.850 INFO:teuthology.orchestra.run.smithi031.stdout:815 read 0x1589dec thru 0x159882e (0xea43 bytes) 2024-09-13T07:12:40.850 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.850 INFO:teuthology.orchestra.run.smithi031.stdout:817 write 0x77feadc thru 0x780ba3e (0xcf63 bytes) 2024-09-13T07:12:40.850 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:40.850 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:40.853 INFO:teuthology.orchestra.run.smithi031.stdout:818 punch from 0x6fb1968 to 0x6fc1467, (0xfaff bytes) 2024-09-13T07:12:40.853 INFO:teuthology.orchestra.run.smithi031.stdout:819 writesame 0x98d0bfa thru 0x98d1191 data_size 0x4(0x598 bytes) 2024-09-13T07:12:40.856 INFO:teuthology.orchestra.run.smithi031.stdout:821 write 0x37efcb thru 0x38a938 (0xb96e bytes) 2024-09-13T07:12:40.860 INFO:teuthology.orchestra.run.smithi031.stdout:822 trunc from 0xb02b9d6 to 0x91267ff 2024-09-13T07:12:40.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:40.869 INFO:teuthology.orchestra.run.smithi031.stdout:824 write 0x1b25ce7 thru 0x1b30cf4 (0xb00e bytes) 2024-09-13T07:12:40.889 INFO:teuthology.orchestra.run.smithi031.stdout:825 writesame 0xeda7371 thru 0xeda7e22 data_size 0x1(0xab2 bytes) 2024-09-13T07:12:40.889 INFO:teuthology.orchestra.run.smithi031.stdout:828 write 0x9f41339 thru 0x9f4452c (0x31f4 bytes) 2024-09-13T07:12:40.892 INFO:teuthology.orchestra.run.smithi031.stdout:830 write 0x27b7ad1 thru 0x27b9a5d (0x1f8d bytes) 2024-09-13T07:12:40.895 INFO:teuthology.orchestra.run.smithi031.stdout:831 write 0xd38159e thru 0xd38b345 (0x9da8 bytes) 2024-09-13T07:12:40.898 INFO:teuthology.orchestra.run.smithi031.stdout:832 read 0xc908f9b thru 0xc90da58 (0x4abe bytes) 2024-09-13T07:12:40.898 INFO:teuthology.orchestra.run.smithi031.stdout:833 writesame 0xd116188 thru 0xd11ca47 data_size 0x20(0x68c0 bytes) 2024-09-13T07:12:40.901 INFO:teuthology.orchestra.run.smithi031.stdout:834 read 0xcb6a39b thru 0xcb78d85 (0xe9eb bytes) 2024-09-13T07:12:40.902 INFO:teuthology.orchestra.run.smithi031.stdout:835 read 0xa5246b4 thru 0xa5316ff (0xd04c bytes) 2024-09-13T07:12:40.902 INFO:teuthology.orchestra.run.smithi031.stdout:836 read 0x9cc32a9 thru 0x9cc51af (0x1f07 bytes) 2024-09-13T07:12:40.902 INFO:teuthology.orchestra.run.smithi031.stdout:837 write 0x96a5066 thru 0x96a517c (0x117 bytes) 2024-09-13T07:12:40.904 INFO:teuthology.orchestra.run.smithi031.stdout:838 write 0x55522fe thru 0x555a299 (0x7f9c bytes) 2024-09-13T07:12:40.907 INFO:teuthology.orchestra.run.smithi031.stdout:839 read 0x14ddd38 thru 0x14e0367 (0x2630 bytes) 2024-09-13T07:12:40.907 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:40.907 INFO:teuthology.orchestra.run.smithi031.stdout:841 read 0x34d2950 thru 0x34db787 (0x8e38 bytes) 2024-09-13T07:12:40.907 INFO:teuthology.orchestra.run.smithi031.stdout:842 read 0x1bb4d9e thru 0x1bb5dff (0x1062 bytes) 2024-09-13T07:12:40.911 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:40.912+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:40.918 INFO:teuthology.orchestra.run.smithi031.stdout:843 trunc from 0xeda7e23 to 0x49d63e1 2024-09-13T07:12:40.931 INFO:teuthology.orchestra.run.smithi031.stdout:844 trunc from 0x49d63e1 to 0x63d1f82 2024-09-13T07:12:40.942 INFO:teuthology.orchestra.run.smithi031.stdout:845 trunc from 0x63d1f82 to 0xb09238a 2024-09-13T07:12:40.957 INFO:teuthology.orchestra.run.smithi031.stdout:846 trunc from 0xb09238a to 0x2ad1395 2024-09-13T07:12:40.979 INFO:teuthology.orchestra.run.smithi031.stdout:847 write 0xb099f63 thru 0xb0a066b (0x6709 bytes) 2024-09-13T07:12:40.988 INFO:teuthology.orchestra.run.smithi031.stdout:848 trunc from 0xb0a066c to 0x56ad211 2024-09-13T07:12:40.992 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:40.993 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:40.998 INFO:teuthology.orchestra.run.smithi031.stdout:849 read 0xe08e4b thru 0xe0c8ad (0x3a63 bytes) 2024-09-13T07:12:40.999 INFO:teuthology.orchestra.run.smithi031.stdout:851 punch from 0x4f6e0ba to 0x4f6ff4f, (0x1e95 bytes) 2024-09-13T07:12:40.999 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:41.000 INFO:teuthology.orchestra.run.smithi031.stdout:855 trunc from 0x56ad211 to 0x6122ed6 2024-09-13T07:12:41.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:41.005 INFO:teuthology.orchestra.run.smithi031.stdout:856 read 0xe935b2 thru 0xe9ac0b (0x765a bytes) 2024-09-13T07:12:41.006 INFO:teuthology.orchestra.run.smithi031.stdout:857 punch from 0x52ad168 to 0x52b644f, (0x92e7 bytes) 2024-09-13T07:12:41.012 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:41.012+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:41.021 INFO:teuthology.orchestra.run.smithi031.stdout:859 write 0xe90889a thru 0xe91033c (0x7aa3 bytes) 2024-09-13T07:12:41.024 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:41.024 INFO:teuthology.orchestra.run.smithi031.stdout:861 write 0x4d79ad8 thru 0x4d88439 (0xe962 bytes) 2024-09-13T07:12:41.030 INFO:teuthology.orchestra.run.smithi031.stdout:863 write 0xb2b9963 thru 0xb2bc47f (0x2b1d bytes) 2024-09-13T07:12:41.042 INFO:teuthology.orchestra.run.smithi031.stdout:864 trunc from 0xe91033d to 0x66fd120 2024-09-13T07:12:41.053 INFO:teuthology.orchestra.run.smithi031.stdout:866 write 0x3b4cad7 thru 0x3b4d5fc (0xb26 bytes) 2024-09-13T07:12:41.056 INFO:teuthology.orchestra.run.smithi031.stdout:870 read 0x60bf38a thru 0x60bff54 (0xbcb bytes) 2024-09-13T07:12:41.056 INFO:teuthology.orchestra.run.smithi031.stdout:871 read 0x50701e7 thru 0x507d99e (0xd7b8 bytes) 2024-09-13T07:12:41.056 INFO:teuthology.orchestra.run.smithi031.stdout:874 punch from 0x5c39b63 to 0x5c48e9f, (0xf33c bytes) 2024-09-13T07:12:41.057 INFO:teuthology.orchestra.run.smithi031.stdout:875 read 0x61bb889 thru 0x61c1138 (0x58b0 bytes) 2024-09-13T07:12:41.057 INFO:teuthology.orchestra.run.smithi031.stdout:876 read 0x3f8f4a0 thru 0x3f9a161 (0xacc2 bytes) 2024-09-13T07:12:41.057 INFO:teuthology.orchestra.run.smithi031.stdout:878 read 0xf01f75 thru 0xf119c7 (0xfa53 bytes) 2024-09-13T07:12:41.060 INFO:teuthology.orchestra.run.smithi031.stdout:880 trunc from 0x66fd120 to 0xa59f4e2 2024-09-13T07:12:41.067 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:41.067 INFO:teuthology.orchestra.run.smithi031.stdout:883 write 0x6020599 thru 0x6023eed (0x3955 bytes) 2024-09-13T07:12:41.070 INFO:teuthology.orchestra.run.smithi031.stdout:884 read 0x97d2712 thru 0x97db9f4 (0x92e3 bytes) 2024-09-13T07:12:41.070 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:41.070 INFO:teuthology.orchestra.run.smithi031.stdout:886 write 0x6bba979 thru 0x6bc2c6d (0x82f5 bytes) 2024-09-13T07:12:41.074 INFO:teuthology.orchestra.run.smithi031.stdout:890 flatten 2024-09-13T07:12:41.083 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:41.083 INFO:teuthology.orchestra.run.smithi031.stdout:892 write 0x954ef87 thru 0x95568a5 (0x791f bytes) 2024-09-13T07:12:41.091 INFO:teuthology.orchestra.run.smithi031.stdout:893 write 0xa92eedd thru 0xa93b490 (0xc5b4 bytes) 2024-09-13T07:12:41.094 INFO:teuthology.orchestra.run.smithi031.stdout:894 writesame 0xa323fcd thru 0xa328dee data_size 0x49(0x4e22 bytes) 2024-09-13T07:12:41.098 INFO:teuthology.orchestra.run.smithi031.stdout:895 writesame 0x608a778 thru 0x609370f data_size 0x14(0x8f98 bytes) 2024-09-13T07:12:41.101 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xe760aa3 2024-09-13T07:12:41.106 INFO:teuthology.orchestra.run.smithi031.stdout:898 trunc from 0xa93b491 to 0xe760aa3 2024-09-13T07:12:41.113 INFO:teuthology.orchestra.run.smithi031.stdout:899 read 0x5fe4c11 thru 0x5fed59d (0x898d bytes) 2024-09-13T07:12:41.113 INFO:teuthology.orchestra.run.smithi031.stdout:900 write 0x7242a63 thru 0x7247e33 (0x53d1 bytes) 2024-09-13T07:12:41.113 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:41.114+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:41.115 INFO:teuthology.orchestra.run.smithi031.stdout:901 punch from 0x95d1473 to 0x95de319, (0xcea6 bytes) 2024-09-13T07:12:41.129 INFO:teuthology.orchestra.run.smithi031.stdout:902 trunc from 0xe760aa3 to 0x229c892 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:913 3.1e 3:7c3d5185:::rbd_data.1173f77ecad8.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.245992+0000", 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8854885700000001, 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052291789999999996, 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 913 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245992+0000", 2024-09-13T07:12:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245992+0000", 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245991+0000", 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.245997+0000", 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246010+0000", 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2422000000000001e-05 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246018+0000", 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0069999999999997e-06 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246075+0000", 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-05 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246158+0000", 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2939e-05 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246372+0000", 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000214198 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248033+0000", 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016612720000000001 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251112+0000", 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030792559999999998 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251172+0000", 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9700000000000001e-05 2024-09-13T07:12:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251221+0000", 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9444000000000003e-05 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:923 3.7 3:e0d0441f:::rbd_data.1173f77ecad8.0000000000000010:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8796953240000001, 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054522629999999997, 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 923 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251784+0000", 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251790+0000", 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8300000000000001e-06 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251802+0000", 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1935e-05 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251807+0000", 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251857+0000", 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9515000000000003e-05 2024-09-13T07:12:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.252009+0000", 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015179700000000001 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.257238+0000", 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052290870000000003 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1018 3.2 3:445402e0:::rbd_data.1173f77ecad8.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271055+0000", 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8604256719999999, 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050128129999999996, 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1018 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271055+0000", 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271055+0000", 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271054+0000", 2024-09-13T07:12:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271068+0000", 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3923999999999999e-05 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271083+0000", 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4755e-05 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271091+0000", 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3510000000000002e-06 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271145+0000", 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4206000000000002e-05 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271224+0000", 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8722999999999998e-05 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271430+0000", 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206565 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.272720+0000", 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012896850000000001 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.275974+0000", 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032543519999999999 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276027+0000", 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379000000000001e-05 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276068+0000", 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1114e-05 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1020 3.8 3:11bc453b:::rbd_data.1173f77ecad8.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271120+0000", 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8603611500000001, 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051601049999999999, 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1020 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271120+0000", 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271120+0000", 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271119+0000", 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271123+0000", 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3340000000000003e-06 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271133+0000", 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2280000000000004e-06 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271139+0000", 2024-09-13T07:12:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.02e-06 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271184+0000", 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5031000000000003e-05 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271262+0000", 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8380999999999994e-05 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271455+0000", 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019298199999999999 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276096+0000", 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046414300000000002 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276191+0000", 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4306000000000001e-05 2024-09-13T07:12:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276241+0000", 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0614e-05 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276280+0000", 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8387999999999999e-05 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1021 3.b 3:d4309e9e:::rbd_data.1173f77ecad8.000000000000002d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271673+0000", 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8598083020000002, 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053003850000000003, 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1021 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271673+0000", 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271673+0000", 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271672+0000", 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271681+0000", 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5909999999999996e-06 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271689+0000", 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271701+0000", 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1857e-05 2024-09-13T07:12:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271767+0000", 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6196000000000001e-05 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271912+0000", 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014517100000000001 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276973+0000", 2024-09-13T07:12:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050606710000000001 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1028 3.8 3:133f0075:::rbd_data.1173f77ecad8.0000000000000031:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.273575+0000", 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.857905605, 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054363240000000002, 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1028 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273575+0000", 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273575+0000", 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273574+0000", 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273579+0000", 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.566e-06 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273590+0000", 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1095e-05 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273595+0000", 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9060000000000001e-06 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273640+0000", 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5694e-05 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273777+0000", 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013655500000000001 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.279012+0000", 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005234486 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1054 3.1e 3:7c990db8:::rbd_data.1173f77ecad8.000000000000003f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.282376+0000", 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8491043629999999, 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049235939999999999, 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1054 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282376+0000", 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282376+0000", 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282375+0000", 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282381+0000", 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3129999999999998e-06 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282394+0000", 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3575e-05 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282402+0000", 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1049999999999995e-06 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282450+0000", 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7074999999999999e-05 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.282581+0000", 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013110500000000001 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.287300+0000", 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047194280000000003 2024-09-13T07:12:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1056 3.b 3:d41379e8:::rbd_data.1173f77ecad8.0000000000000041:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.284900+0000", 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8465809220000002, 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053290769999999998, 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1056 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284900+0000", 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284900+0000", 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284899+0000", 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284904+0000", 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1730000000000003e-06 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284920+0000", 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6113e-05 2024-09-13T07:12:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284928+0000", 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7479999999999993e-06 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.284973+0000", 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5024000000000001e-05 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.285115+0000", 2024-09-13T07:12:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014217200000000001 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290229+0000", 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051138709999999999 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1075 3.b 3:d41379e8:::rbd_data.1173f77ecad8.0000000000000041:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8410198659999999, 2024-09-13T07:12:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057904439999999996, 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290460+0000", 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290466+0000", 2024-09-13T07:12:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290477+0000", 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1595e-05 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290485+0000", 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7360000000000001e-06 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290527+0000", 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1690000000000002e-05 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290610+0000", 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3321999999999999e-05 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290800+0000", 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019058399999999999 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293341+0000", 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025410139999999999 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296160+0000", 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028181460000000001 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296209+0000", 2024-09-13T07:12:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9055999999999999e-05 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296251+0000", 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2703000000000001e-05 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1098 3.15 e25/19 3:afeba560:::rbd_data.1173f77ecad8.000000000000004f:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8320183439999997, 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091051389999999999, 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6639999999999998e-06 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299484+0000", 2024-09-13T07:12:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4609e-05 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299491+0000", 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.584e-06 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299534+0000", 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640999999999998e-05 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299553+0000", 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8564999999999999e-05 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308532+0000", 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089789510000000006 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308568+0000", 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5896000000000003e-05 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.c epoch 25/19 rep_tid 48 entries 25'5 (0'0) error 3:32647bc6:::rbd_data.1173f77ecad8.0000000000000057:head by client.4467.0:1100 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'2)", 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8302203590000001, 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073257039999999997, 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301266+0000", 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5239999999999997e-06 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301279+0000", 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2856e-05 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301296+0000", 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6858e-05 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301369+0000", 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2631000000000003e-05 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308586+0000", 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072174989999999996 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1104 3.1e 3:7e73e8c7:::rbd_data.1173f77ecad8.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8296188989999997, 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009729985, 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1104 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301861+0000", 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301866+0000", 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301879+0000", 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3083e-05 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301893+0000", 2024-09-13T07:12:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4656e-05 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301950+0000", 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6671e-05 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302027+0000", 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6859000000000004e-05 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302260+0000", 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023318300000000001 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303132+0000", 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000871487 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311492+0000", 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083604839999999996 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311550+0000", 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7701000000000002e-05 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311592+0000", 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2024000000000003e-05 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1106 3.c e25/19 3:30d6dbcc:::rbd_data.1173f77ecad8.0000000000000050:head v 25'6, mlcod=25'3)", 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8290391019999999, 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090851709999999995, 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302441+0000", 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302447+0000", 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7969999999999997e-06 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302460+0000", 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2772000000000001e-05 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302466+0000", 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302507+0000", 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1014999999999999e-05 2024-09-13T07:12:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302520+0000", 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2706e-05 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311485+0000", 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089648929999999998 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311527+0000", 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2259000000000003e-05 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1118 3.1e 3:7d8c2ee8:::rbd_data.1173f77ecad8.000000000000005f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.822461712, 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056840500000000004, 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1118 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309018+0000", 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309023+0000", 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0950000000000003e-06 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309034+0000", 2024-09-13T07:12:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0857e-05 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309041+0000", 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0639999999999996e-06 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309094+0000", 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2936000000000003e-05 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309241+0000", 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146618 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.314703+0000", 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054623320000000003 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1117 3.c e25/19 3:32647bc6:::rbd_data.1173f77ecad8.0000000000000057:head v 25'7, mlcod=25'4)", 2024-09-13T07:12:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8220536009999999, 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052532430000000003, 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309427+0000", 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309433+0000", 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6730000000000003e-06 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309449+0000", 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5299999999999999e-05 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309454+0000", 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309495+0000", 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1285000000000002e-05 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309518+0000", 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2963e-05 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.314644+0000", 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051258409999999999 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.314680+0000", 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6650000000000003e-05 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1130 3.1e 3:7b48162d:::rbd_data.1173f77ecad8.000000000000005c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8192695209999998, 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060295670000000004, 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1130 2024-09-13T07:12:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312210+0000", 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312217+0000", 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.939e-06 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312229+0000", 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2269000000000001e-05 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312237+0000", 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8310000000000001e-06 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312306+0000", 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8515000000000004e-05 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312412+0000", 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312643+0000", 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023097699999999999 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.317531+0000", 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048874529999999999 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318137+0000", 2024-09-13T07:12:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000606352 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318196+0000", 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8816000000000001e-05 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318241+0000", 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5170999999999998e-05 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.5 epoch 25/19 rep_tid 53 entries 25'6 (0'0) error 3:a62f531c:::rbd_data.1173f77ecad8.0000000000000062:head by client.4467.0:1129 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'4)", 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8191023820000001, 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051462469999999996, 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312378+0000", 2024-09-13T07:12:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312385+0000", 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0539999999999997e-06 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312396+0000", 2024-09-13T07:12:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1208e-05 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312404+0000", 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312453+0000", 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8915999999999997e-05 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.317525+0000", 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050714610000000002 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1154 3.b 3:d3009503:::rbd_data.1173f77ecad8.0000000000000168:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.803825003, 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059705729999999999, 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1154 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327655+0000", 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327661+0000", 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6860000000000003e-06 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327672+0000", 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1809e-05 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327680+0000", 2024-09-13T07:12:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9400000000000002e-06 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327729+0000", 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9023000000000001e-05 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327807+0000", 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7768999999999994e-05 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327961+0000", 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000153616 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.330853+0000", 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002891897 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333519+0000", 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026667359999999998 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333576+0000", 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6786000000000001e-05 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333626+0000", 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0144e-05 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1155 3.b 3:d723b608:::rbd_data.1173f77ecad8.00000000000001d6:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8033394930000002, 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058401670000000003, 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1155 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328145+0000", 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5820000000000003e-06 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328156+0000", 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1255e-05 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328163+0000", 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0149999999999998e-06 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328502+0000", 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033881500000000001 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328582+0000", 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166000000000005e-05 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328777+0000", 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019505799999999999 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333240+0000", 2024-09-13T07:12:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044629070000000003 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333900+0000", 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065926699999999995 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333943+0000", 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3027e-05 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333981+0000", 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8828999999999997e-05 2024-09-13T07:12:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1828 3.18 3:1c28841c:::rbd_data.1173c53667c.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write 963576~17461] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.985083098, 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085613430000000008, 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1828 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146396+0000", 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146417+0000", 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1087999999999999e-05 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146431+0000", 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3788000000000001e-05 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146442+0000", 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0545e-05 2024-09-13T07:12:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146535+0000", 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3067999999999997e-05 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146542+0000", 2024-09-13T07:12:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5290000000000003e-06 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148549+0000", 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002006407 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148675+0000", 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126549 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148818+0000", 2024-09-13T07:12:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014280900000000001 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.152082+0000", 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032638709999999998 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154899+0000", 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028167520000000001 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154935+0000", 2024-09-13T07:12:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6392e-05 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154959+0000", 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4170000000000001e-05 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:41.236 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:41.238 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:41.239 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xec83b4b 2024-09-13T07:12:41.239 INFO:teuthology.orchestra.run.smithi031.stdout:903 trunc from 0x229c892 to 0xec83b4b 2024-09-13T07:12:41.239 INFO:teuthology.orchestra.run.smithi031.stdout:904 write 0x9accff thru 0x9af7a2 (0x2aa4 bytes) 2024-09-13T07:12:41.239 INFO:teuthology.orchestra.run.smithi031.stdout:905 writesame 0xc8c72fe thru 0xc8d3aaf data_size 0x1(0xc7b2 bytes) 2024-09-13T07:12:41.239 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:12:41.239 INFO:teuthology.orchestra.run.smithi031.stdout:906 writesame 0x3d65260 thru 0x3d70b16 data_size 0x1(0xb8b7 bytes) 2024-09-13T07:12:41.240 INFO:teuthology.orchestra.run.smithi031.stdout:908 writesame 0xdba2644 thru 0xdba604e data_size 0xd(0x3a0b bytes) 2024-09-13T07:12:41.241 INFO:teuthology.orchestra.run.smithi031.stdout:910 write 0x188dea5 thru 0x189c169 (0xe2c5 bytes) 2024-09-13T07:12:41.241 INFO:teuthology.orchestra.run.smithi031.stdout:911 punch from 0xb91dc75 to 0xb91ebd2, (0xf5d bytes) 2024-09-13T07:12:41.241 INFO:teuthology.orchestra.run.smithi031.stdout:913 writesame 0x302c9a0 thru 0x3030445 data_size 0x1(0x3aa6 bytes) 2024-09-13T07:12:41.241 INFO:teuthology.orchestra.run.smithi031.stdout:915 read 0x2ef39b thru 0x2f527f (0x5ee5 bytes) 2024-09-13T07:12:41.241 INFO:teuthology.orchestra.run.smithi031.stdout:916 write 0xba5a2e thru 0xbb567e (0xfc51 bytes) 2024-09-13T07:12:41.241 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:41.238+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:41.243 INFO:teuthology.orchestra.run.smithi031.stdout:917 trunc from 0xec83b4b to 0x938d354 2024-09-13T07:12:41.252 INFO:teuthology.orchestra.run.smithi031.stdout:919 punch from 0x66cd31a to 0x66d5bb0, (0x8896 bytes) 2024-09-13T07:12:41.252 INFO:teuthology.orchestra.run.smithi031.stdout:920 writesame 0x27ac911 thru 0x27b6eda data_size 0x1(0xa5ca bytes) 2024-09-13T07:12:41.272 INFO:teuthology.orchestra.run.smithi031.stdout:922 punch from 0x56cc2ec to 0x56d5911, (0x9625 bytes) 2024-09-13T07:12:41.272 INFO:teuthology.orchestra.run.smithi031.stdout:924 punch from 0x812c949 to 0x81389e7, (0xc09e bytes) 2024-09-13T07:12:41.282 INFO:teuthology.orchestra.run.smithi031.stdout:926 trunc from 0x938d354 to 0x218403 2024-09-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi031.stdout:927 trunc from 0x218403 to 0x46ceab4 2024-09-13T07:12:41.320 INFO:teuthology.orchestra.run.smithi031.stdout:928 write 0xae3db58 thru 0xae4b9ba (0xde63 bytes) 2024-09-13T07:12:41.324 INFO:teuthology.orchestra.run.smithi031.stdout:929 read 0x1bf9fe5 thru 0x1bff18b (0x51a7 bytes) 2024-09-13T07:12:41.324 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:41.324 INFO:teuthology.orchestra.run.smithi031.stdout:932 read 0x3d18d3e thru 0x3d221e3 (0x94a6 bytes) 2024-09-13T07:12:41.324 INFO:teuthology.orchestra.run.smithi031.stdout:934 punch from 0x782b4e to 0x7845cf, (0x1a81 bytes) 2024-09-13T07:12:41.324 INFO:teuthology.orchestra.run.smithi031.stdout:936 write 0x1f96f2 thru 0x201322 (0x7c31 bytes) 2024-09-13T07:12:41.327 INFO:teuthology.orchestra.run.smithi031.stdout:938 read 0x2acedf3 thru 0x2ade204 (0xf412 bytes) 2024-09-13T07:12:41.332 INFO:teuthology.orchestra.run.smithi031.stdout:939 trunc from 0xae4b9bb to 0xe8d8074 2024-09-13T07:12:41.337 INFO:teuthology.orchestra.run.smithi031.stdout:940 read 0x3327569 thru 0x3327a06 (0x49e bytes) 2024-09-13T07:12:41.338 INFO:teuthology.orchestra.run.smithi031.stdout:941 writesame 0xbb0ba77 thru 0xbb0fdbc data_size 0x4f(0x4346 bytes) 2024-09-13T07:12:41.338 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:41.339+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:41.340 INFO:teuthology.orchestra.run.smithi031.stdout:942 read 0xdb41317 thru 0xdb42713 (0x13fd bytes) 2024-09-13T07:12:41.346 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:41.346 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:41.346 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:41.346 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:41.355 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:943 trunc from 0xe8d8074 to 0x17a302 2024-09-13T07:12:41.367 INFO:teuthology.orchestra.run.smithi031.stdout:944 read 0xa8c5e thru 0xa8d74 (0x117 bytes) 2024-09-13T07:12:41.367 INFO:teuthology.orchestra.run.smithi031.stdout:945 punch from 0x51537 to 0x5b891, (0xa35a bytes) 2024-09-13T07:12:41.367 INFO:teuthology.orchestra.run.smithi031.stdout:947 read 0x13bee1 thru 0x13d501 (0x1621 bytes) 2024-09-13T07:12:41.368 INFO:teuthology.orchestra.run.smithi031.stdout:948 punch from 0x10585d to 0x10c879, (0x701c bytes) 2024-09-13T07:12:41.380 INFO:teuthology.orchestra.run.smithi031.stdout:949 write 0x620dadd thru 0x6214d84 (0x72a8 bytes) 2024-09-13T07:12:41.388 INFO:teuthology.orchestra.run.smithi031.stdout:950 writesame 0x6b2868d thru 0x6b34656 data_size 0x93(0xbfca bytes) 2024-09-13T07:12:41.388 INFO:teuthology.orchestra.run.smithi031.stdout:952 read 0xa8e0ac thru 0xa90785 (0x26da bytes) 2024-09-13T07:12:41.394 INFO:teuthology.orchestra.run.smithi031.stdout:954 write 0x7f66884 thru 0x7f693bf (0x2b3c bytes) 2024-09-13T07:12:41.397 INFO:teuthology.orchestra.run.smithi031.stdout:955 write 0x54ff7 thru 0x5f2fc (0xa306 bytes) 2024-09-13T07:12:41.399 INFO:teuthology.orchestra.run.smithi031.stdout:957 read 0x5225aa2 thru 0x5232713 (0xcc72 bytes) 2024-09-13T07:12:41.399 INFO:teuthology.orchestra.run.smithi031.stdout:958 read 0x52690ca thru 0x526e8d6 (0x580d bytes) 2024-09-13T07:12:41.399 INFO:teuthology.orchestra.run.smithi031.stdout:959 read 0x40f3d97 thru 0x40f877b (0x49e5 bytes) 2024-09-13T07:12:41.400 INFO:teuthology.orchestra.run.smithi031.stdout:961 trunc from 0x7f693c0 to 0x8fc9721 2024-09-13T07:12:41.406 INFO:teuthology.orchestra.run.smithi031.stdout:962 punch from 0xa1c47 to 0xafd5d, (0xe116 bytes) 2024-09-13T07:12:41.414 INFO:teuthology.orchestra.run.smithi031.stdout:963 write 0x9a4b9bc thru 0x9a4f5bc (0x3c01 bytes) 2024-09-13T07:12:41.417 INFO:teuthology.orchestra.run.smithi031.stdout:964 read 0x35b11e2 thru 0x35bcb53 (0xb972 bytes) 2024-09-13T07:12:41.421 INFO:teuthology.orchestra.run.smithi031.stdout:965 trunc from 0x9a4f5bd to 0x59fa8a5 2024-09-13T07:12:41.432 INFO:teuthology.orchestra.run.smithi031.stdout:967 read 0x428ded1 thru 0x42988b0 (0xa9e0 bytes) 2024-09-13T07:12:41.433 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:41.439 INFO:teuthology.orchestra.run.smithi031.stdout:971 write 0x677d756 thru 0x6782e04 (0x56af bytes) 2024-09-13T07:12:41.439 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:41.440+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:41.456 INFO:teuthology.orchestra.run.smithi031.stdout:974 writesame 0xe14e804 thru 0xe155e95 data_size 0x3(0x7692 bytes) 2024-09-13T07:12:41.456 INFO:teuthology.orchestra.run.smithi031.stdout:975 clone 5 order 19 su 16384 sc 8 2024-09-13T07:12:41.498 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:41.498 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:41.498 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:41.498 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:41.498 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:41.507 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:41.539 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:41.541+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:41.640 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:41.641+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:791 3.14 3:2ad4b4d4:::rbd_data.1173f77ecad8.0000000000000147:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:36.639691+0000", 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0121153100000004, 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064195340000000002, 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 791 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639691+0000", 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639691+0000", 2024-09-13T07:12:41.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639690+0000", 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639693+0000", 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.399e-06 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639703+0000", 2024-09-13T07:12:41.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.611e-06 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639708+0000", 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4060000000000004e-06 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639740+0000", 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2020999999999997e-05 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.639821+0000", 2024-09-13T07:12:41.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1137000000000006e-05 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.640784+0000", 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00096226000000000005 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.645971+0000", 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051877240000000003 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.646012+0000", 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0537999999999997e-05 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.646062+0000", 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0489999999999999e-05 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:36.646110+0000", 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7574000000000002e-05 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:914 3.14 3:2ed78eca:::rbd_data.1173f77ecad8.000000000000000a:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.4053851689999997, 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082734499999999999, 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 914 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246420+0000", 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246424+0000", 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246430+0000", 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8000000000000001e-06 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246434+0000", 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:12:41.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246477+0000", 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2443e-05 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246632+0000", 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015543100000000001 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254694+0000", 2024-09-13T07:12:41.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080618979999999996 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:919 3.1a 3:5d4b50e3:::rbd_data.1173f77ecad8.000000000000000d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.248431+0000", 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.4033750319999996, 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060058289999999999, 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:41.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 919 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248431+0000", 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248431+0000", 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248430+0000", 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248434+0000", 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2089999999999999e-06 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248446+0000", 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1715999999999999e-05 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248454+0000", 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3790000000000006e-06 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248482+0000", 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7778999999999999e-05 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248554+0000", 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2334000000000002e-05 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254437+0000", 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058821510000000004 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "pg_update_log_missing(3.1 epoch 25/19 rep_tid 36 entries 25'20 (0'0) error 3:87900178:::rbd_data.1173f77ecad8.000000000000000e:head by client.4467.0:920 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'18)", 2024-09-13T07:12:41.666 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.248944+0000", 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.4028623109999998, 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054260560000000003, 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248944+0000", 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248944+0000", 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248943+0000", 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248950+0000", 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.81e-06 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248963+0000", 2024-09-13T07:12:41.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2836e-05 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248977+0000", 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4039999999999999e-05 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.249018+0000", 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1362999999999997e-05 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254370+0000", 2024-09-13T07:12:41.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005351682 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1128 3.3 3:c2b5a8ac:::rbd_data.1173f77ecad8.0000000000000061:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.311928+0000", 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.339878004, 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061792139999999997, 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.670 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1128 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311928+0000", 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311928+0000", 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311927+0000", 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311932+0000", 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311943+0000", 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0625000000000001e-05 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311954+0000", 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0679000000000001e-05 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311998+0000", 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.401e-05 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312088+0000", 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0781999999999999e-05 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.318107+0000", 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060187640000000002 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1151 3.9 3:9280c38f:::rbd_data.1173f77ecad8.000000000000010c:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3249451600000004, 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069283519999999996, 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.674 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1151 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:41.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326860+0000", 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326863+0000", 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326872+0000", 2024-09-13T07:12:41.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2199999999999992e-06 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326886+0000", 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4066999999999999e-05 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327486+0000", 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00060069599999999996 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327560+0000", 2024-09-13T07:12:41.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3709999999999997e-05 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327669+0000", 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108544 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.330996+0000", 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033276099999999999 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:41.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333730+0000", 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002733311 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333767+0000", 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7861000000000002e-05 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333789+0000", 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1608999999999999e-05 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1153 3.1b e25/19 3:d88532a6:::rbd_data.1173f77ecad8.000000000000015c:head v 25'14, mlcod=25'11)", 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3242383039999996, 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091902019999999997, 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327567+0000", 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327572+0000", 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327579+0000", 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.306e-06 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327585+0000", 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1279999999999996e-06 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327625+0000", 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9227999999999998e-05 2024-09-13T07:12:41.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327641+0000", 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6059999999999999e-05 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336708+0000", 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090672949999999995 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336758+0000", 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9870000000000002e-05 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1551 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'89, mlcod=25'86)", 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8735810009999998, 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077485289999999997, 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778224+0000", 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778233+0000", 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4689999999999997e-06 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778245+0000", 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2097e-05 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778252+0000", 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.934e-06 2024-09-13T07:12:41.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778288+0000", 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6012e-05 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778302+0000", 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4630999999999999e-05 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785938+0000", 2024-09-13T07:12:41.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076354329999999996 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785973+0000", 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5663000000000001e-05 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1552 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'90, mlcod=25'86)", 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:41.688 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8729327360000001, 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0071221499999999998, 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778887+0000", 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4409999999999999e-05 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778897+0000", 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0344e-05 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778903+0000", 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7019999999999998e-06 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778933+0000", 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0358e-05 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778955+0000", 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1795999999999999e-05 2024-09-13T07:12:41.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785976+0000", 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070212269999999997 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785995+0000", 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8907e-05 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1553 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'91, mlcod=25'86)", 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.779472+0000", 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8723339389999998, 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065460170000000003, 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779472+0000", 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779472+0000", 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779471+0000", 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779475+0000", 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.07e-06 2024-09-13T07:12:41.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779482+0000", 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8850000000000001e-06 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779487+0000", 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6090000000000003e-06 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779513+0000", 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5843e-05 2024-09-13T07:12:41.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.779530+0000", 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6889e-05 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785997+0000", 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064669389999999997 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786018+0000", 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1335000000000002e-05 2024-09-13T07:12:41.696 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1554 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'92, mlcod=25'86)", 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.780024+0000", 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8717821269999999, 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060148739999999999, 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780024+0000", 2024-09-13T07:12:41.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780024+0000", 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780023+0000", 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780029+0000", 2024-09-13T07:12:41.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5840000000000002e-06 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780039+0000", 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0743e-05 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780045+0000", 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9120000000000003e-06 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780078+0000", 2024-09-13T07:12:41.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2286999999999998e-05 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780094+0000", 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6041000000000001e-05 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786019+0000", 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059255449999999999 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786039+0000", 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9403999999999999e-05 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1555 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'93, mlcod=25'86)", 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8711489139999999, 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0081821140000000007, 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780656+0000", 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780661+0000", 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.079e-06 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780670+0000", 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6030000000000005e-06 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780676+0000", 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780720+0000", 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3767000000000003e-05 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780739+0000", 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9504000000000002e-05 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788759+0000", 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080197340000000006 2024-09-13T07:12:41.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788839+0000", 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0075999999999997e-05 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1556 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'94, mlcod=25'86)", 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8704721559999999, 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0075812830000000003, 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.705 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781333+0000", 2024-09-13T07:12:41.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781337+0000", 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781344+0000", 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781349+0000", 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781376+0000", 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7826000000000001e-05 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781392+0000", 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5401999999999999e-05 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788842+0000", 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007450684 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788915+0000", 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2556000000000001e-05 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1557 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'95, mlcod=25'86)", 2024-09-13T07:12:41.709 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8698809230000002, 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070583310000000002, 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:41.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781928+0000", 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4699999999999998e-06 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781934+0000", 2024-09-13T07:12:41.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.302e-06 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781938+0000", 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2559999999999998e-06 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781967+0000", 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9856e-05 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781979+0000", 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1857e-05 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788917+0000", 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069380750000000001 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788983+0000", 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5906000000000002e-05 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.713 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1558 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'96, mlcod=25'87)", 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.782560+0000", 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8692459079999999, 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064554690000000001, 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782560+0000", 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782560+0000", 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782559+0000", 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782564+0000", 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8289999999999998e-06 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782571+0000", 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0679999999999999e-06 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782578+0000", 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7190000000000004e-06 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782612+0000", 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3918000000000003e-05 2024-09-13T07:12:41.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782623+0000", 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1326e-05 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788985+0000", 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0063617020000000003 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789015+0000", 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0454999999999999e-05 2024-09-13T07:12:41.717 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1559 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'97, mlcod=25'87)", 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.783107+0000", 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.8686988850000001, 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059300250000000002, 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783107+0000", 2024-09-13T07:12:41.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783107+0000", 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783106+0000", 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783110+0000", 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8249999999999998e-06 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783117+0000", 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783121+0000", 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7979999999999999e-06 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783151+0000", 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9873e-05 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783161+0000", 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0403e-05 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789017+0000", 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058556729999999996 2024-09-13T07:12:41.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789037+0000", 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.014e-05 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:2859 3.9 3:946400fb:::rbd_data.11734c6039c3.000000000000006d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.034120951, 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019805990999999998, 2024-09-13T07:12:41.722 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 2859 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617683+0000", 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617695+0000", 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1626e-05 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617723+0000", 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8252999999999999e-05 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617737+0000", 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3993e-05 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617824+0000", 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6908000000000002e-05 2024-09-13T07:12:41.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617958+0000", 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013417 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.621384+0000", 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034260499999999999 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637421+0000", 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016037264999999998 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637430+0000", 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1549999999999996e-06 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637462+0000", 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1958000000000001e-05 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637491+0000", 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8523000000000001e-05 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.727 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:2859 3.9 e27/19 ondisk, result = 0)", 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:40.631140+0000", 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.020665795, 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062834689999999999, 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:41.728 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.631140+0000", 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.631140+0000", 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.631139+0000", 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.631143+0000", 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8750000000000002e-06 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.631154+0000", 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195e-05 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.631162+0000", 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637409+0000", 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062476349999999996 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637417+0000", 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.729e-06 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637424+0000", 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.63e-06 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3313 3.1a 3:5f0039c6:::rbd_data.11734c6039c3.000000000000003e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.45406212299999998, 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021783180999999999, 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3313 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197742+0000", 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197754+0000", 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2519e-05 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197797+0000", 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3028999999999997e-05 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197803+0000", 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7420000000000003e-06 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197870+0000", 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7123000000000001e-05 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197959+0000", 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8409e-05 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.201633+0000", 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036746299999999999 2024-09-13T07:12:41.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219455+0000", 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017821452000000002 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219464+0000", 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8309999999999996e-06 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219498+0000", 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3948000000000001e-05 2024-09-13T07:12:41.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219527+0000", 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8461000000000001e-05 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:3313 3.1a e28/19 ondisk, result = 0)", 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.212445+0000", 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.43936121500000003, 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007012964, 2024-09-13T07:12:41.737 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212445+0000", 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212445+0000", 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212444+0000", 2024-09-13T07:12:41.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212448+0000", 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212461+0000", 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2798e-05 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212469+0000", 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4739999999999999e-06 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219441+0000", 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069722050000000004 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219450+0000", 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2089999999999994e-06 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219458+0000", 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4619999999999999e-06 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.740 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.741 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.741 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:41.741 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:41.741 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:41.742 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:41.743 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:41.743+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:41.843 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:41.844+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:41.845 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:41.845 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:41.845 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:41.845 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:41.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:41.944 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:41.945+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:41.996 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:41.996 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:41.996 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:41.996 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:41.996 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:42.005 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:42.045 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:42.045+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:42.108 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1298 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.646598268, 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01046212, 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1298 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461385+0000", 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461390+0000", 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5210000000000004e-06 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461394+0000", 2024-09-13T07:12:42.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2230000000000001e-06 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461397+0000", 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6910000000000002e-06 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464234+0000", 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028368820000000002 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464318+0000", 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4067000000000007e-05 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464664+0000", 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034640300000000001 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468940+0000", 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042761190000000001 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471764+0000", 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002823739 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471807+0000", 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3485999999999997e-05 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471848+0000", 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0392999999999998e-05 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.114 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1300 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6465650790000002, 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010517047999999999, 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1300 2024-09-13T07:12:42.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461418+0000", 2024-09-13T07:12:42.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461421+0000", 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3489999999999999e-06 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461425+0000", 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8330000000000004e-06 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461427+0000", 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5519999999999999e-06 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465053+0000", 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036258309999999999 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465104+0000", 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0825999999999998e-05 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465312+0000", 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020849900000000001 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468969+0000", 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036562119999999998 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471860+0000", 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002891841 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471903+0000", 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2315999999999999e-05 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471936+0000", 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3132000000000001e-05 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1301 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6465347389999998, 2024-09-13T07:12:42.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010564656, 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1301 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461452+0000", 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4690000000000002e-06 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461456+0000", 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0810000000000004e-06 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461459+0000", 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465574+0000", 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041147930000000003 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465655+0000", 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1254999999999994e-05 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466023+0000", 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036823900000000002 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468992+0000", 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029688190000000001 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471945+0000", 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029525609999999998 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471982+0000", 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7837999999999999e-05 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472014+0000", 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1432e-05 2024-09-13T07:12:42.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1302 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6465045199999997, 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010611844, 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1302 2024-09-13T07:12:42.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461481+0000", 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4040000000000002e-06 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461485+0000", 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7969999999999998e-06 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461488+0000", 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6400000000000001e-06 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466476+0000", 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049885140000000003 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466555+0000", 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8208999999999996e-05 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466936+0000", 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038099899999999998 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469019+0000", 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020838300000000001 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472021+0000", 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030016019999999999 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472056+0000", 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5295e-05 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472091+0000", 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4876999999999998e-05 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1303 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6464722250000001, 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010655628, 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1303 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.132 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461511+0000", 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461514+0000", 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.418e-06 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461518+0000", 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9219999999999999e-06 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461520+0000", 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.57e-06 2024-09-13T07:12:42.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467284+0000", 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057634100000000001 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467336+0000", 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2633999999999998e-05 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467664+0000", 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00032754700000000003 2024-09-13T07:12:42.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471710+0000", 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004046231 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472100+0000", 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038953300000000001 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472136+0000", 2024-09-13T07:12:42.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6699000000000002e-05 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472167+0000", 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1019999999999998e-05 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1304 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6464471200000004, 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010707311000000001, 2024-09-13T07:12:42.137 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1304 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461536+0000", 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461539+0000", 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.244e-06 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461543+0000", 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7099999999999998e-06 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461546+0000", 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5170000000000002e-06 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468072+0000", 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065260650000000002 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468137+0000", 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5036999999999994e-05 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468389+0000", 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025219400000000002 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471754+0000", 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033649280000000001 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000421183 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472212+0000", 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7153999999999997e-05 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472244+0000", 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1612999999999997e-05 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1305 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6464159609999998, 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010819396, 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1305 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461570+0000", 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3860000000000001e-06 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461574+0000", 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461577+0000", 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6340000000000001e-06 2024-09-13T07:12:42.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469041+0000", 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0074637030000000004 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469091+0000", 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0197000000000001e-05 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469298+0000", 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020718900000000001 2024-09-13T07:12:42.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472253+0000", 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002954702 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472306+0000", 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3198999999999997e-05 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472351+0000", 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000001e-05 2024-09-13T07:12:42.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472387+0000", 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6145e-05 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1306 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6463852919999997, 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010918835999999999, 2024-09-13T07:12:42.148 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1306 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461598+0000", 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461602+0000", 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0860000000000004e-06 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461610+0000", 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0830000000000004e-06 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461616+0000", 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7379999999999999e-06 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469601+0000", 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0079851690000000003 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469669+0000", 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8316000000000001e-05 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469866+0000", 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000196768 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472259+0000", 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002392786 2024-09-13T07:12:42.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472441+0000", 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018190200000000001 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472485+0000", 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3853999999999997e-05 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472517+0000", 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2614999999999999e-05 2024-09-13T07:12:42.153 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1307 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.646340125, 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0110288, 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.154 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1307 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461643+0000", 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461647+0000", 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9469999999999996e-06 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461656+0000", 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9279999999999999e-06 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461664+0000", 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5989999999999996e-06 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470152+0000", 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0084883609999999998 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470233+0000", 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1287999999999998e-05 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470632+0000", 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00039872099999999999 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472531+0000", 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018986420000000001 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472593+0000", 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2902000000000002e-05 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472637+0000", 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000002e-05 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472673+0000", 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5839e-05 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1308 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.159 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6458965169999997, 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011378667, 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1308 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462086+0000", 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462092+0000", 2024-09-13T07:12:42.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9020000000000004e-06 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462104+0000", 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1807999999999999e-05 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462113+0000", 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7560000000000005e-06 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470891+0000", 2024-09-13T07:12:42.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008778569 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470955+0000", 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3859e-05 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471342+0000", 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038666200000000002 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473305+0000", 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019625630000000001 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473376+0000", 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1354999999999997e-05 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473424+0000", 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8010000000000003e-05 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473466+0000", 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2132000000000003e-05 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1309 3.1d 3:bcefd1cd:::rbd_data.1173c53667c.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.6442380779999999, 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014288309000000001, 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1309 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463745+0000", 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463751+0000", 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463761+0000", 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3589999999999997e-06 2024-09-13T07:12:42.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463767+0000", 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4420000000000003e-06 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472731+0000", 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0089641479999999999 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472858+0000", 2024-09-13T07:12:42.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012669100000000001 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473057+0000", 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019875900000000001 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477793+0000", 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047361900000000004 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477983+0000", 2024-09-13T07:12:42.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019018299999999999 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478014+0000", 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1062999999999997e-05 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478034+0000", 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.978e-05 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1373 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5847576219999997, 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010609936, 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1373 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523231+0000", 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9280000000000001e-06 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523238+0000", 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3509999999999996e-06 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523243+0000", 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0900000000000004e-06 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.527993+0000", 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047498699999999998 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.528077+0000", 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3616999999999996e-05 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.528470+0000", 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000393853 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532078+0000", 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036080019999999999 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533743+0000", 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001664998 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533790+0000", 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6326000000000003e-05 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533836+0000", 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6490999999999999e-05 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1378 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5840977540000001, 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010563678, 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.176 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1378 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523889+0000", 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2499999999999998e-06 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523896+0000", 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523901+0000", 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8679999999999998e-06 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532169+0000", 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082678690000000006 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532251+0000", 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2207999999999996e-05 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532620+0000", 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036913700000000001 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534210+0000", 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015899690000000001 2024-09-13T07:12:42.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534377+0000", 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016627800000000001 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534416+0000", 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9626000000000003e-05 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534450+0000", 2024-09-13T07:12:42.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3482999999999998e-05 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1379 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5839714599999999, 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011548345999999999, 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.182 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1379 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524016+0000", 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.703e-06 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524026+0000", 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-13T07:12:42.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524031+0000", 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9190000000000004e-06 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532975+0000", 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008944044 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533061+0000", 2024-09-13T07:12:42.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5353999999999996e-05 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533391+0000", 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00032971500000000001 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534330+0000", 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093939800000000003 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535481+0000", 2024-09-13T07:12:42.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011505700000000001 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535522+0000", 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1958e-05 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535561+0000", 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8278999999999997e-05 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1380 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5837670859999999, 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012748349000000001, 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1380 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524216+0000", 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524221+0000", 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4519999999999996e-06 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524233+0000", 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1652e-05 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524241+0000", 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1100000000000003e-06 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534487+0000", 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010246207 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534570+0000", 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2948999999999995e-05 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534908+0000", 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00033817199999999997 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536427+0000", 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015188420000000001 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536868+0000", 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044121999999999998 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536920+0000", 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1520999999999997e-05 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536965+0000", 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5160999999999997e-05 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1555 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3304896910000004, 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011519636, 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.193 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1555 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:42.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777497+0000", 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5609999999999999e-06 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777504+0000", 2024-09-13T07:12:42.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5220000000000004e-06 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777509+0000", 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7890000000000002e-06 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780260+0000", 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0027512460000000002 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780343+0000", 2024-09-13T07:12:42.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3100000000000001e-05 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780598+0000", 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025544100000000002 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786194+0000", 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055953080000000002 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788934+0000", 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002739806 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788978+0000", 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4274000000000003e-05 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789014+0000", 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6010999999999998e-05 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1556 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3304697680000004, 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011620324, 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1556 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777517+0000", 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7980000000000001e-06 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777524+0000", 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2899999999999997e-06 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777529+0000", 2024-09-13T07:12:42.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8319999999999996e-06 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781008+0000", 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034795770000000002 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781090+0000", 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1499e-05 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781297+0000", 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000207689 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786431+0000", 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051334140000000002 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789071+0000", 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026401670000000001 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789103+0000", 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2400999999999997e-05 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789134+0000", 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0967999999999999e-05 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1557 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.204 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3304294089999997, 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011717251999999999, 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1557 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777558+0000", 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2869999999999998e-06 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777568+0000", 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7680000000000003e-06 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777574+0000", 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6520000000000003e-06 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781632+0000", 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040581599999999999 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781679+0000", 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6934000000000002e-05 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781867+0000", 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018818900000000001 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786435+0000", 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045683039999999996 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789187+0000", 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002752151 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789232+0000", 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.494e-05 2024-09-13T07:12:42.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789272+0000", 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9376999999999999e-05 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1558 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3303892839999998, 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011767046999999999, 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.210 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1558 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:42.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777594+0000", 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777597+0000", 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2140000000000001e-06 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777605+0000", 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777610+0000", 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782276+0000", 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046664000000000002 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782321+0000", 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4489e-05 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782507+0000", 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018627699999999999 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786438+0000", 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039313780000000001 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789315+0000", 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002876871 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789342+0000", 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6415e-05 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789362+0000", 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9868999999999999e-05 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.215 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1559 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3303000110000003, 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011750661000000001, 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1559 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.216 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777683+0000", 2024-09-13T07:12:42.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777690+0000", 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8730000000000001e-06 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777703+0000", 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3016999999999999e-05 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777710+0000", 2024-09-13T07:12:42.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782759+0000", 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050484909999999996 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782845+0000", 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6082999999999996e-05 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783061+0000", 2024-09-13T07:12:42.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000215732 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788822+0000", 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057617689999999999 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789389+0000", 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00056687899999999995 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789416+0000", 2024-09-13T07:12:42.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7132999999999999e-05 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789435+0000", 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8199999999999999e-05 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.221 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:42.223 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:42.223 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:42.223+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:42.324 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:42.325+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:42.326 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:42.326 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:42.326 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:42.326 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:42.335 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:42.393 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T07:12:42.425 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:42.426+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:42.480 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:42.481 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:42.489 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:42.526 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:42.526+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:42.546 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.546 INFO:teuthology.orchestra.run.smithi031.stdout:977 read 0x5beb17e thru 0x5beef77 (0x3dfa bytes) 2024-09-13T07:12:42.546 INFO:teuthology.orchestra.run.smithi031.stdout:978 read 0xd8cfdef thru 0xd8d9b11 (0x9d23 bytes) 2024-09-13T07:12:42.547 INFO:teuthology.orchestra.run.smithi031.stdout:980 writesame 0xbe037c5 thru 0xbe11280 data_size 0x2(0xdabc bytes) 2024-09-13T07:12:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:981 read 0x5ed7890 thru 0x5ee5897 (0xe008 bytes) 2024-09-13T07:12:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:982 write 0x1ac67b thru 0x1b3900 (0x7286 bytes) 2024-09-13T07:12:42.573 INFO:teuthology.orchestra.run.smithi031.stdout:983 writesame 0x656e68f thru 0x6577e53 data_size 0x9(0x97c5 bytes) 2024-09-13T07:12:42.584 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.584 INFO:teuthology.orchestra.run.smithi031.stdout:985 write 0x8172976 thru 0x817e95e (0xbfe9 bytes) 2024-09-13T07:12:42.591 INFO:teuthology.orchestra.run.smithi031.stdout:987 read 0x6ad369b thru 0x6ad46d0 (0x1036 bytes) 2024-09-13T07:12:42.591 INFO:teuthology.orchestra.run.smithi031.stdout:988 punch from 0xe517ef to 0xe59293, (0x7aa4 bytes) 2024-09-13T07:12:42.591 INFO:teuthology.orchestra.run.smithi031.stdout:993 write 0x539e9d8 thru 0x53a77a1 (0x8dca bytes) 2024-09-13T07:12:42.598 INFO:teuthology.orchestra.run.smithi031.stdout:994 trunc from 0xe155e96 to 0xbc6f1cb 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4257.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.007719843, 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036526549999999999, 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.603 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594197+0000", 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594211+0000", 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.399e-05 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594220+0000", 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2359999999999993e-06 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594231+0000", 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0293e-05 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594280+0000", 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9261000000000002e-05 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594301+0000", 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0744999999999999e-05 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597812+0000", 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035110950000000001 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597850+0000", 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:908 3.10 3:09fecc17:::rbd_data.1173f77ecad8.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:42.607 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3576694329999999, 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040361349999999997, 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 908 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:42.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244247+0000", 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244253+0000", 2024-09-13T07:12:42.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6389999999999997e-06 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244264+0000", 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1544e-05 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244270+0000", 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244309+0000", 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8838999999999999e-05 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244385+0000", 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6097e-05 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244591+0000", 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000206749 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246079+0000", 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014876679999999999 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248230+0000", 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021511780000000001 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248264+0000", 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3283e-05 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248284+0000", 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0598000000000002e-05 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:913 3.1e e25/19 3:7c3d5185:::rbd_data.1173f77ecad8.0000000000000005:head v 25'4, mlcod=24'2)", 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3553112030000003, 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004381462, 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.613 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:42.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246612+0000", 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9499999999999998e-06 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246622+0000", 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0217000000000001e-05 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246627+0000", 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0540000000000002e-06 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5330000000000002e-05 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246684+0000", 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1645e-05 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.250954+0000", 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042697819999999997 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.250988+0000", 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4215999999999999e-05 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:922 3.a 3:52856a69:::rbd_data.1173f77ecad8.000000000000000f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.617 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3505456870000003, 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058310970000000004, 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 922 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251371+0000", 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251376+0000", 2024-09-13T07:12:42.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9450000000000001e-06 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251387+0000", 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0786e-05 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251392+0000", 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3310000000000003e-06 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251426+0000", 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3767999999999999e-05 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251503+0000", 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7410000000000006e-05 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.257203+0000", 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056996900000000003 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.621 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1010 3.10 3:0d2ccec7:::rbd_data.1173f77ecad8.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3326922650000004, 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042914140000000003, 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1010 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269224+0000", 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269230+0000", 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7030000000000003e-06 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269243+0000", 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3215e-05 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269248+0000", 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0889999999999999e-06 2024-09-13T07:12:42.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269292+0000", 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3928999999999999e-05 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269370+0000", 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8163000000000004e-05 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269581+0000", 2024-09-13T07:12:42.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021031700000000001 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.270174+0000", 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059290199999999997 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273429+0000", 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00325524 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273485+0000", 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5926e-05 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273517+0000", 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1826000000000003e-05 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1052 3.10 3:0982ea78:::rbd_data.1173f77ecad8.000000000000003e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3197054650000002, 2024-09-13T07:12:42.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052248729999999997, 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1052 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282211+0000", 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282217+0000", 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6350000000000003e-06 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282228+0000", 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1323e-05 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282245+0000", 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6807e-05 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282279+0000", 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4100000000000002e-05 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282365+0000", 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6012000000000003e-05 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287437+0000", 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050720519999999996 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.631 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1061 3.10 3:0c912c88:::rbd_data.1173f77ecad8.0000000000000044:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3167949190000003, 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005098455, 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1061 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:42.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285122+0000", 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285126+0000", 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3340000000000003e-06 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285135+0000", 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9039999999999998e-06 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285142+0000", 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285181+0000", 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8717000000000002e-05 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285304+0000", 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012349400000000001 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290221+0000", 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049169610000000001 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.635 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1067 3.10 3:0982ea78:::rbd_data.1173f77ecad8.000000000000003e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3143315250000001, 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035721809999999998, 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1067 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.636 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287585+0000", 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287590+0000", 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2560000000000004e-06 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287599+0000", 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3349999999999996e-06 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287604+0000", 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5470000000000004e-06 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287646+0000", 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1992e-05 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287714+0000", 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7594000000000004e-05 2024-09-13T07:12:42.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.288113+0000", 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00039889400000000001 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.288598+0000", 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000484981 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291095+0000", 2024-09-13T07:12:42.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024973759999999999 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291137+0000", 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1987999999999998e-05 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291158+0000", 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0919000000000001e-05 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1073 3.10 3:09473a54:::rbd_data.1173f77ecad8.0000000000000049:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3115387619999996, 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004417575, 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1073 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290378+0000", 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290383+0000", 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3179999999999997e-06 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290396+0000", 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2492e-05 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290402+0000", 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.759e-06 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290439+0000", 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6915000000000001e-05 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290530+0000", 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0853999999999994e-05 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.294796+0000", 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042662339999999998 2024-09-13T07:12:42.645 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1074 3.10 3:0c912c88:::rbd_data.1173f77ecad8.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3112321329999999, 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055866580000000004, 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1074 2024-09-13T07:12:42.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290691+0000", 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0800000000000002e-06 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290699+0000", 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7390000000000005e-06 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290704+0000", 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7280000000000003e-06 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290731+0000", 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7299000000000001e-05 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290776+0000", 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4848000000000002e-05 2024-09-13T07:12:42.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290902+0000", 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012560100000000001 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.294803+0000", 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039015579999999998 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296179+0000", 2024-09-13T07:12:42.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013752879999999999 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296230+0000", 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1270999999999998e-05 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296272+0000", 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2035e-05 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1125 3.a 3:5135e61f:::rbd_data.1173f77ecad8.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2904011400000002, 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00420258, 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1125 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311520+0000", 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8919999999999999e-06 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311531+0000", 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0805000000000001e-05 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311538+0000", 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1400000000000002e-06 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311577+0000", 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8395000000000001e-05 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311664+0000", 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7609999999999996e-05 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311792+0000", 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012732499999999999 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315178+0000", 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033866970000000001 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315667+0000", 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00048881399999999996 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315698+0000", 2024-09-13T07:12:42.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0321999999999999e-05 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315719+0000", 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1369000000000002e-05 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1130 3.1e e25/19 3:7b48162d:::rbd_data.1173f77ecad8.000000000000005c:head v 25'13, mlcod=25'10)", 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2891661320000001, 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005273247, 2024-09-13T07:12:42.657 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312757+0000", 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9410000000000004e-06 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312769+0000", 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2799999999999999e-05 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312776+0000", 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8199999999999999e-06 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312819+0000", 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2293999999999999e-05 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312838+0000", 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9123e-05 2024-09-13T07:12:42.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.317986+0000", 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051480340000000001 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.318025+0000", 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8914e-05 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1320 3.10 3:0e76b587:::rbd_data.1173c53667c.0000000000000057:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 497989] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.472830+0000", 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.129087341, 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033972469999999999, 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1320 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472830+0000", 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472830+0000", 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472828+0000", 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472837+0000", 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9609999999999994e-06 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472856+0000", 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8556e-05 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472865+0000", 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9590000000000001e-06 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472924+0000", 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8947999999999999e-05 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473047+0000", 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012370999999999999 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473239+0000", 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000191375 2024-09-13T07:12:42.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.475070+0000", 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018309540000000001 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.476166+0000", 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010964289999999999 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.476205+0000", 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.879e-05 2024-09-13T07:12:42.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.476227+0000", 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2402999999999999e-05 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1348 3.10 3:0e76b587:::rbd_data.1173c53667c.0000000000000057:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0848794020000003, 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038489119999999999, 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.667 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1348 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517036+0000", 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517046+0000", 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7370000000000001e-06 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517066+0000", 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9658999999999999e-05 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517076+0000", 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0217999999999999e-05 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517134+0000", 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7779999999999999e-05 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517218+0000", 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4738000000000001e-05 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517335+0000", 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116573 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.519614+0000", 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022788830000000002 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520800+0000", 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001186063 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520846+0000", 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6161000000000001e-05 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520887+0000", 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0960000000000001e-05 2024-09-13T07:12:42.672 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1536 3.a 3:5448ec73:::rbd_data.1173c53667c.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.83640034, 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004075932, 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.673 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1536 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765516+0000", 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765521+0000", 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0499999999999999e-06 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765532+0000", 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1138000000000001e-05 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765538+0000", 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765575+0000", 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7283000000000002e-05 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765630+0000", 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4818999999999998e-05 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765695+0000", 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5215000000000005e-05 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766354+0000", 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00065942400000000003 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769502+0000", 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031472549999999998 2024-09-13T07:12:42.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769551+0000", 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9577999999999999e-05 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769593+0000", 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1983000000000001e-05 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1542 3.a 3:52be2a25:::rbd_data.1173c53667c.0000000000000043:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8322145990000003, 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005091641, 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1542 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769702+0000", 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769708+0000", 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3509999999999998e-06 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769722+0000", 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4334e-05 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769730+0000", 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7330000000000003e-06 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769789+0000", 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9077999999999999e-05 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769860+0000", 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0832000000000002e-05 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769978+0000", 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118257 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.770500+0000", 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052145399999999997 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774714+0000", 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042146079999999999 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774760+0000", 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.583e-05 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774795+0000", 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4288000000000001e-05 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5647251879999997, 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.684 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.685 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.689 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1768 3.a e25/19 ondisk, result = 0)", 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5498521030000001, 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010131984, 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052064+0000", 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052072+0000", 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9529999999999999e-06 2024-09-13T07:12:42.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052090+0000", 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8263000000000001e-05 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052102+0000", 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2078e-05 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062047+0000", 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0099446580000000003 2024-09-13T07:12:42.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062059+0000", 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2455e-05 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062197+0000", 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013813200000000001 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1787 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.108204+0000", 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.4937134150000002, 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003334264, 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1787 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108204+0000", 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108204+0000", 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108201+0000", 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999971 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108212+0000", 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0631e-05 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108234+0000", 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1673999999999999e-05 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108244+0000", 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9890000000000004e-06 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108302+0000", 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8207e-05 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108388+0000", 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5661000000000005e-05 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.108499+0000", 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000111098 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.109169+0000", 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00067065600000000005 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.111404+0000", 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022343129999999999 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.111482+0000", 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8257000000000006e-05 2024-09-13T07:12:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.111538+0000", 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6499999999999998e-05 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3345 3.10 3:0be6394f:::rbd_data.11734c6039c3.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.315804003, 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035746290000000002, 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3345 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286111+0000", 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999971 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286123+0000", 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2471e-05 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286146+0000", 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3136999999999999e-05 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286156+0000", 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4550000000000001e-06 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286216+0000", 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0245999999999999e-05 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:12:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286223+0000", 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1590000000000004e-06 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288195+0000", 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001972279 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288274+0000", 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8882999999999997e-05 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288352+0000", 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8158999999999995e-05 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289021+0000", 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00066851300000000003 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289615+0000", 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059398999999999997 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289657+0000", 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1989000000000001e-05 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289688+0000", 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1257000000000002e-05 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:42.705 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:42.707 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:12:42.708 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:42.707+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:996 punch from 0x34a228a to 0x34b15a9, (0xf31f bytes) 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:998 write 0x97bd587 thru 0x97c1a42 (0x44bc bytes) 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:999 writesame 0x9cdd221 thru 0x9cdd3b9 data_size 0x1(0x199 bytes) 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:1000 write 0xb8cfd46 thru 0xb8d2c9f (0x2f5a bytes) 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:1002 writesame 0xa7aeb9f thru 0xa7bdca7 data_size 0x2b(0xf109 bytes) 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:1004 writesame 0xba2e506 thru 0xba32bf3 data_size 0x7(0x46ee bytes) 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:1005 punch from 0x5777c34 to 0x57837d3, (0xbb9f bytes) 2024-09-13T07:12:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:1006 write 0xcf8f120 thru 0xcf92bf6 (0x3ad7 bytes) 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:1007 write 0x46ff700 thru 0x4707185 (0x7a86 bytes) 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:1009 punch from 0x329c1c5 to 0x32a8466, (0xc2a1 bytes) 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:1011 write 0xb085cce thru 0xb08a9fd (0x4d30 bytes) 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:1013 trunc from 0xcf92bf7 to 0x42fdff2 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:1016 read 0x1620edb thru 0x16303d6 (0xf4fc bytes) 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:1018 write 0x95698d4 thru 0x9571c7f (0x83ac bytes) 2024-09-13T07:12:42.710 INFO:teuthology.orchestra.run.smithi031.stdout:1019 trunc from 0x9571c80 to 0xbef4956 2024-09-13T07:12:42.712 INFO:teuthology.orchestra.run.smithi031.stdout:1020 punch from 0xa03b4fc to 0xa03df36, (0x2a3a bytes) 2024-09-13T07:12:42.712 INFO:teuthology.orchestra.run.smithi031.stdout:1021 read 0x87fe87e thru 0x880bc1e (0xd3a1 bytes) 2024-09-13T07:12:42.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:12:42.718 INFO:teuthology.orchestra.run.smithi031.stdout:1024 write 0xc52e183 thru 0xc53a2e4 (0xc162 bytes) 2024-09-13T07:12:42.723 INFO:teuthology.orchestra.run.smithi031.stdout:1025 read 0x8db182b thru 0x8db3d33 (0x2509 bytes) 2024-09-13T07:12:42.723 INFO:teuthology.orchestra.run.smithi031.stdout:1027 writesame 0x8d452c5 thru 0x8d4efb6 data_size 0x1(0x9cf2 bytes) 2024-09-13T07:12:42.733 INFO:teuthology.orchestra.run.smithi031.stdout:1028 write 0x8c44775 thru 0x8c46aad (0x2339 bytes) 2024-09-13T07:12:42.735 INFO:teuthology.orchestra.run.smithi031.stdout:1030 writesame 0xbd333ef thru 0xbd38ccd data_size 0x1(0x58df bytes) 2024-09-13T07:12:42.748 INFO:teuthology.orchestra.run.smithi031.stdout:1032 punch from 0x7ea4238 to 0x7ea50f3, (0xebb bytes) 2024-09-13T07:12:42.748 INFO:teuthology.orchestra.run.smithi031.stdout:1033 read 0xb3de8b3 thru 0xb3e7554 (0x8ca2 bytes) 2024-09-13T07:12:42.748 INFO:teuthology.orchestra.run.smithi031.stdout:1034 read 0x67db516 thru 0x67dcc8b (0x1776 bytes) 2024-09-13T07:12:42.755 INFO:teuthology.orchestra.run.smithi031.stdout:1035 trunc from 0xc53a2e5 to 0x5ee87dc 2024-09-13T07:12:42.779 INFO:teuthology.orchestra.run.smithi031.stdout:1036 writesame 0x670148a thru 0x6706b01 data_size 0x4(0x5678 bytes) 2024-09-13T07:12:42.780 INFO:teuthology.orchestra.run.smithi031.stdout:1037 trunc from 0x6706b02 to 0x5233afb 2024-09-13T07:12:42.785 INFO:teuthology.orchestra.run.smithi031.stdout:1040 writesame 0x305570 thru 0x308932 data_size 0x7(0x33c3 bytes) 2024-09-13T07:12:42.792 INFO:teuthology.orchestra.run.smithi031.stdout:1041 write 0x1ba124f thru 0x1ba13d9 (0x18b bytes) 2024-09-13T07:12:42.794 INFO:teuthology.orchestra.run.smithi031.stdout:1042 read 0x221099b thru 0x22111fa (0x860 bytes) 2024-09-13T07:12:42.795 INFO:teuthology.orchestra.run.smithi031.stdout:1043 trunc from 0x5233afb to 0x571000a 2024-09-13T07:12:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:1046 read 0x53d1cc9 thru 0x53d7194 (0x54cc bytes) 2024-09-13T07:12:42.807 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:42.808+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:42.811 INFO:teuthology.orchestra.run.smithi031.stdout:1048 write 0xa78747e thru 0xa78e2f6 (0x6e79 bytes) 2024-09-13T07:12:42.819 INFO:teuthology.orchestra.run.smithi031.stdout:1049 read 0x6aee27 thru 0x6afd2e (0xf08 bytes) 2024-09-13T07:12:42.819 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.820 INFO:teuthology.orchestra.run.smithi031.stdout:1051 trunc from 0xa78e2f7 to 0x930226d 2024-09-13T07:12:42.839 INFO:teuthology.orchestra.run.smithi031.stdout:1052 write 0xb239d71 thru 0xb23c94c (0x2bdc bytes) 2024-09-13T07:12:42.845 INFO:teuthology.orchestra.run.smithi031.stdout:1053 punch from 0x3692719 to 0x369fdf2, (0xd6d9 bytes) 2024-09-13T07:12:42.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.845 INFO:teuthology.orchestra.run.smithi031.stdout:1057 read 0xa0bcef7 thru 0xa0c2bd2 (0x5cdc bytes) 2024-09-13T07:12:42.845 INFO:teuthology.orchestra.run.smithi031.stdout:1058 writesame 0x6549c84 thru 0x6552248 data_size 0xf(0x85c5 bytes) 2024-09-13T07:12:42.853 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:42.853 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:42.854 INFO:teuthology.orchestra.run.smithi031.stdout:1059 read 0x878f8c6 thru 0x8790b51 (0x128c bytes) 2024-09-13T07:12:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:1061 trunc from 0xb23c94d to 0x87cf243 2024-09-13T07:12:42.860 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:42.861 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:42.861 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:42.861 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:42.870 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:12:42.871 INFO:teuthology.orchestra.run.smithi031.stdout:1062 punch from 0x404b27d to 0x4056e4e, (0xbbd1 bytes) 2024-09-13T07:12:42.871 INFO:teuthology.orchestra.run.smithi031.stdout:1063 read 0x55a7b3a thru 0x55b28e8 (0xadaf bytes) 2024-09-13T07:12:42.871 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:42.871 INFO:teuthology.orchestra.run.smithi031.stdout:1065 trunc from 0x87cf243 to 0x7fee987 2024-09-13T07:12:42.886 INFO:teuthology.orchestra.run.smithi031.stdout:1066 write 0xc5321cd thru 0xc5394ae (0x72e2 bytes) 2024-09-13T07:12:42.891 INFO:teuthology.orchestra.run.smithi031.stdout:1067 writesame 0x9f35136 thru 0x9f38663 data_size 0x3(0x352e bytes) 2024-09-13T07:12:42.896 INFO:teuthology.orchestra.run.smithi031.stdout:1068 writesame 0x661bfbc thru 0x6626c91 data_size 0x1(0xacd6 bytes) 2024-09-13T07:12:42.908 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:42.909+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:42.916 INFO:teuthology.orchestra.run.smithi031.stdout:1069 writesame 0x13f3f9e thru 0x13f9eb6 data_size 0x5(0x5f19 bytes) 2024-09-13T07:12:42.934 INFO:teuthology.orchestra.run.smithi031.stdout:1070 trunc from 0xc5394af to 0x2d021f1 2024-09-13T07:12:42.961 INFO:teuthology.orchestra.run.smithi031.stdout:1071 punch from 0x210424e to 0x210a318, (0x60ca bytes) 2024-09-13T07:12:42.961 INFO:teuthology.orchestra.run.smithi031.stdout:1072 trunc from 0x2d021f1 to 0x2d4bd62 2024-09-13T07:12:42.972 INFO:teuthology.orchestra.run.smithi031.stdout:1074 trunc from 0x2d4bd62 to 0x922c348 2024-09-13T07:12:42.978 INFO:teuthology.orchestra.run.smithi031.stdout:1075 read 0x69f39b4 thru 0x69f944c (0x5a99 bytes) 2024-09-13T07:12:42.978 INFO:teuthology.orchestra.run.smithi031.stdout:1076 read 0x3e55fea thru 0x3e6297f (0xc996 bytes) 2024-09-13T07:12:42.978 INFO:teuthology.orchestra.run.smithi031.stdout:1077 punch from 0x49ed7a2 to 0x49eef8b, (0x17e9 bytes) 2024-09-13T07:12:42.979 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:42.979 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:42.979 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:42.979 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:42.979 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:42.985 INFO:teuthology.orchestra.run.smithi031.stdout:1078 write 0xb0ab6ad thru 0xb0b2ef0 (0x7844 bytes) 2024-09-13T07:12:42.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:12:42.996 INFO:teuthology.orchestra.run.smithi031.stdout:1081 write 0xc83106c thru 0xc83deae (0xce43 bytes) 2024-09-13T07:12:43.006 INFO:teuthology.orchestra.run.smithi031.stdout:1082 write 0x2ed9a9f thru 0x2ee9232 (0xf794 bytes) 2024-09-13T07:12:43.009 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:43.010+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:43.023 INFO:teuthology.orchestra.run.smithi031.stdout:1083 trunc from 0xc83deaf to 0xd32809 2024-09-13T07:12:43.047 INFO:teuthology.orchestra.run.smithi031.stdout:1084 read 0x68e736 thru 0x69a98e (0xc259 bytes) 2024-09-13T07:12:43.048 INFO:teuthology.orchestra.run.smithi031.stdout:1085 trunc from 0xd32809 to 0x18cc421 2024-09-13T07:12:43.052 INFO:teuthology.orchestra.run.smithi031.stdout:1086 punch from 0xaec14d to 0xafb9f8, (0xf8ab bytes) 2024-09-13T07:12:43.075 INFO:teuthology.orchestra.run.smithi031.stdout:1088 writesame 0xdb03f07 thru 0xdb0ad55 data_size 0x1(0x6e4f bytes) 2024-09-13T07:12:43.075 INFO:teuthology.orchestra.run.smithi031.stdout:1089 read 0x2fae172 thru 0x2fb4632 (0x64c1 bytes) 2024-09-13T07:12:43.087 INFO:teuthology.orchestra.run.smithi031.stdout:1091 trunc from 0xdb0ad56 to 0x3f90eb1 2024-09-13T07:12:43.088 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:43.093 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.100 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.592431+0000", 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.508016131000002, 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.23577988, 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592431+0000", 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592431+0000", 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592430+0000", 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592443+0000", 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2893e-05 2024-09-13T07:12:43.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592453+0000", 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0022e-05 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592462+0000", 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8740000000000001e-06 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592512+0000", 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9713999999999999e-05 2024-09-13T07:12:43.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.592519+0000", 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7300000000000005e-06 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.824875+0000", 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.23235576399999999 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.825012+0000", 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013701499999999999 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.825213+0000", 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020039600000000001 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.826125+0000", 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00091261399999999996 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.828120+0000", 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019948209999999999 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.828170+0000", 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9599999999999999e-05 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.828211+0000", 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1557999999999998e-05 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:34 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.234092238999999, 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092232949999999994, 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.108 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 34 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866355+0000", 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866359+0000", 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5569999999999997e-06 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866366+0000", 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9079999999999998e-06 2024-09-13T07:12:43.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866381+0000", 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5231999999999999e-05 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868233+0000", 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001851204 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868283+0000", 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0065999999999998e-05 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868365+0000", 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2354999999999993e-05 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.870405+0000", 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00203981 2024-09-13T07:12:43.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875520+0000", 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005114749 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875544+0000", 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4709e-05 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875579+0000", 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4139e-05 2024-09-13T07:12:43.113 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:35 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866391+0000", 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.234056843000001, 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092673200000000008, 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 35 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866391+0000", 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866391+0000", 2024-09-13T07:12:43.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866390+0000", 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866467+0000", 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6987e-05 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866476+0000", 2024-09-13T07:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7690000000000002e-06 2024-09-13T07:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866482+0000", 2024-09-13T07:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7780000000000004e-06 2024-09-13T07:12:43.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868635+0000", 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002153313 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868697+0000", 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1642000000000004e-05 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.868784+0000", 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7052000000000006e-05 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.871584+0000", 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028000019999999998 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875588+0000", 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040048439999999996 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875626+0000", 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7209000000000002e-05 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875658+0000", 2024-09-13T07:12:43.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2255000000000002e-05 2024-09-13T07:12:43.121 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:36 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866490+0000", 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.233957845999999, 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092203830000000004, 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.122 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866490+0000", 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866490+0000", 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866489+0000", 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866494+0000", 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.843e-06 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866501+0000", 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866506+0000", 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:12:43.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869175+0000", 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026688409999999999 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869225+0000", 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0080000000000002e-05 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869316+0000", 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1478999999999996e-05 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.871620+0000", 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023035299999999998 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875662+0000", 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004041958 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875688+0000", 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5972000000000001e-05 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875710+0000", 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2257000000000001e-05 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.129 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:37 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.233936273000001, 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092567640000000007, 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.130 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 37 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866511+0000", 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866515+0000", 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4730000000000002e-06 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866525+0000", 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9289999999999993e-06 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.866530+0000", 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869598+0000", 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030676240000000001 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869675+0000", 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6878000000000005e-05 2024-09-13T07:12:43.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.869771+0000", 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6249000000000006e-05 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.871648+0000", 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018768509999999999 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875715+0000", 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040667530000000002 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875747+0000", 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2759000000000003e-05 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.875768+0000", 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0517999999999999e-05 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:88 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:21.928561+0000", 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.171886752999999, 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092238470000000003, 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.137 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928561+0000", 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928561+0000", 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928560+0000", 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928565+0000", 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3199999999999999e-06 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928572+0000", 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9879999999999998e-06 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.928577+0000", 2024-09-13T07:12:43.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6539999999999998e-06 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.932060+0000", 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034830500000000001 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.932131+0000", 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0581999999999996e-05 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.932209+0000", 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8356999999999995e-05 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.933087+0000", 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087820599999999997 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.937733+0000", 2024-09-13T07:12:43.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046452459999999996 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.937762+0000", 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9271000000000001e-05 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:21.937785+0000", 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2728e-05 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.143 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087833+0000", 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.012614888000002, 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097462420000000004, 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.144 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 112 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087833+0000", 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087833+0000", 2024-09-13T07:12:43.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087832+0000", 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087872+0000", 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9971000000000001e-05 2024-09-13T07:12:43.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087885+0000", 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.326e-05 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087891+0000", 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9070000000000004e-06 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.091686+0000", 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037949720000000002 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.091750+0000", 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4138000000000002e-05 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.091830+0000", 2024-09-13T07:12:43.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9639000000000001e-05 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094491+0000", 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002661358 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097523+0000", 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030311090000000001 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097554+0000", 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1501000000000002e-05 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097579+0000", 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4803e-05 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087900+0000", 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.01254784, 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097636200000000006, 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 113 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.151 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087900+0000", 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087900+0000", 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087899+0000", 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087904+0000", 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087911+0000", 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1829999999999997e-06 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087917+0000", 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8320000000000002e-06 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092280+0000", 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043633170000000002 2024-09-13T07:12:43.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092362+0000", 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1786999999999995e-05 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092458+0000", 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5909999999999995e-05 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094495+0000", 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020376050000000001 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097615+0000", 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00312007 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097642+0000", 2024-09-13T07:12:43.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6548999999999999e-05 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097663+0000", 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1433e-05 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.157 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087921+0000", 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.012526823999998, 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.009817961, 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 114 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087921+0000", 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087921+0000", 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087920+0000", 2024-09-13T07:12:43.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087924+0000", 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.619e-06 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087928+0000", 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.608e-06 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087930+0000", 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9490000000000001e-06 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.092982+0000", 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050519069999999996 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093025+0000", 2024-09-13T07:12:43.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2460999999999999e-05 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093103+0000", 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7645000000000006e-05 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094499+0000", 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013963510000000001 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097695+0000", 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031961619999999998 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097720+0000", 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4737999999999999e-05 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097739+0000", 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8839e-05 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.012510042999999, 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0098704410000000006, 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.164 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 115 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087937+0000", 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087978+0000", 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0407999999999997e-05 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087983+0000", 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3519999999999997e-06 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087986+0000", 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.242e-06 2024-09-13T07:12:43.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093315+0000", 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005329154 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093366+0000", 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1045999999999999e-05 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093440+0000", 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3283999999999998e-05 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.095499+0000", 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020593959999999998 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097768+0000", 2024-09-13T07:12:43.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022686889999999999 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097791+0000", 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2940999999999999e-05 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097808+0000", 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7189000000000001e-05 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.170 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.012457368, 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0098795619999999997, 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.171 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 116 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087990+0000", 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087992+0000", 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4399999999999999e-06 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087995+0000", 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1949999999999999e-06 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.087998+0000", 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.864e-06 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093821+0000", 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058222739999999997 2024-09-13T07:12:43.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093870+0000", 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9008999999999998e-05 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.093954+0000", 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4053000000000003e-05 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.095504+0000", 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00155086 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097828+0000", 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00232402 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097851+0000", 2024-09-13T07:12:43.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2704000000000001e-05 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.097870+0000", 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8476999999999999e-05 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.177 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 21.012444981000002, 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012635965000000001, 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 117 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088002+0000", 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088006+0000", 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8670000000000004e-06 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088012+0000", 2024-09-13T07:12:43.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1600000000000003e-06 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.088015+0000", 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7379999999999999e-06 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094191+0000", 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061757119999999999 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094240+0000", 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8758999999999999e-05 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.094319+0000", 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9567000000000006e-05 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.095719+0000", 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001399939 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.100525+0000", 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048062790000000001 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.100616+0000", 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0410999999999999e-05 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.100638+0000", 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2734999999999998e-05 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.184 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 20.591271295999999, 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010945933, 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 139 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509176+0000", 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509183+0000", 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3030000000000002e-06 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509198+0000", 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4927e-05 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509204+0000", 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1430000000000003e-06 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516097+0000", 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0068931130000000002 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516152+0000", 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5059999999999998e-05 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516244+0000", 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2021999999999994e-05 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.518141+0000", 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018971649999999999 2024-09-13T07:12:43.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520064+0000", 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001923052 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520097+0000", 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2635000000000002e-05 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520122+0000", 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5072999999999999e-05 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 20.591232383000001, 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011295105, 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 140 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509215+0000", 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509274+0000", 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9419999999999997e-05 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509284+0000", 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0293e-05 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509291+0000", 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4860000000000002e-06 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516552+0000", 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072612099999999997 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516632+0000", 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9682e-05 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.516764+0000", 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013198800000000001 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.518146+0000", 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013828340000000001 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520433+0000", 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022862109999999998 2024-09-13T07:12:43.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520476+0000", 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2799000000000001e-05 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.520510+0000", 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4684999999999997e-05 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.197 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4210.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:22.509300+0000", 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 20.591147498000002, 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014418491, 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4210", 2024-09-13T07:12:43.198 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.184:0/2494166651", 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 141 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509300+0000", 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509300+0000", 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509299+0000", 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509305+0000", 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3090000000000003e-06 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509314+0000", 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7540000000000001e-06 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.509320+0000", 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8370000000000002e-06 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517748+0000", 2024-09-13T07:12:43.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0084274450000000004 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517803+0000", 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5894999999999999e-05 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.517899+0000", 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5884000000000006e-05 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.518705+0000", 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00080585599999999998 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.523539+0000", 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048333739999999997 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.523665+0000", 2024-09-13T07:12:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000126811 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:22.523718+0000", 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3084999999999997e-05 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1098 3.15 3:afeba560:::rbd_data.1173f77ecad8.000000000000004f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.299219+0000", 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.8012282229999999, 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096353610000000003, 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1098 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299219+0000", 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299219+0000", 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299219+0000", 2024-09-13T07:12:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299223+0000", 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3429999999999998e-06 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299233+0000", 2024-09-13T07:12:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7790000000000007e-06 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299238+0000", 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3460000000000001e-06 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299267+0000", 2024-09-13T07:12:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8952999999999999e-05 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299309+0000", 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2125000000000001e-05 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.299411+0000", 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00010218 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.300693+0000", 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001281763 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.308773+0000", 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080802900000000004 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.308818+0000", 2024-09-13T07:12:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4644999999999997e-05 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.308855+0000", 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6650999999999999e-05 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1334 3.13 3:cb585f3b:::rbd_data.1173c53667c.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.487900+0000", 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.612547116, 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021810559, 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1334 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.487900+0000", 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.487900+0000", 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.487899+0000", 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.487907+0000", 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6689999999999995e-06 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.487923+0000", 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:12:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.487929+0000", 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.584e-06 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.487970+0000", 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0983999999999999e-05 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.488082+0000", 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011189800000000001 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.491614+0000", 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035322449999999998 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.509635+0000", 2024-09-13T07:12:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.018021074000000002 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.509645+0000", 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6590000000000002e-06 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.509679+0000", 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4391999999999999e-05 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.509711+0000", 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1538000000000002e-05 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1405 3.15 3:aa2b4c04:::rbd_data.1173c53667c.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.547927+0000", 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5525200789999998, 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.026990839999999999, 2024-09-13T07:12:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1405 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.547927+0000", 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.547927+0000", 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.547926+0000", 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.547933+0000", 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6340000000000001e-06 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.547947+0000", 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3145999999999999e-05 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.547953+0000", 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.547998+0000", 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4252999999999998e-05 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.548097+0000", 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9802000000000004e-05 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.551931+0000", 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003833838 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.574776+0000", 2024-09-13T07:12:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.022844496999999998 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.574790+0000", 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4181e-05 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.574851+0000", 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0853999999999997e-05 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.574918+0000", 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7424999999999999e-05 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1405 3.15 e25/19 ondisk, result = 0)", 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.565521+0000", 2024-09-13T07:12:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5349262670000003, 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0092585329999999993, 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.565521+0000", 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.565521+0000", 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.565520+0000", 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.565526+0000", 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6120000000000004e-06 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.565543+0000", 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6662000000000001e-05 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.565554+0000", 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1406999999999999e-05 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.574754+0000", 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091995589999999995 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.574765+0000", 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1386000000000001e-05 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.574780+0000", 2024-09-13T07:12:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4398e-05 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1418 3.13 3:cd9a5063:::rbd_data.1173c53667c.0000000000000096:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.593750+0000", 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5066971210000002, 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021931668000000001, 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1418 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593750+0000", 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593750+0000", 2024-09-13T07:12:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593749+0000", 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593757+0000", 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8830000000000006e-06 2024-09-13T07:12:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593769+0000", 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2075999999999999e-05 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593775+0000", 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1639999999999998e-06 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593822+0000", 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7027999999999998e-05 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.593964+0000", 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000142476 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.597137+0000", 2024-09-13T07:12:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031731400000000001 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.615568+0000", 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.018430716 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.615580+0000", 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2278e-05 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.615633+0000", 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2908999999999998e-05 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.615682+0000", 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8619999999999999e-05 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:43.237 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:43.238 INFO:teuthology.orchestra.run.smithi031.stdout:1093 write 0x9f2c20f thru 0x9f2d669 (0x145b bytes) 2024-09-13T07:12:43.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:43.239 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:43.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:43.240 INFO:teuthology.orchestra.run.smithi031.stdout:1094 read 0x2479914 thru 0x248219f (0x888c bytes) 2024-09-13T07:12:43.240 INFO:teuthology.orchestra.run.smithi031.stdout:1096 write 0x7f7c127 thru 0x7f818b0 (0x578a bytes) 2024-09-13T07:12:43.240 INFO:teuthology.orchestra.run.smithi031.stdout:1097 read 0x54b792e thru 0x54c22d6 (0xa9a9 bytes) 2024-09-13T07:12:43.240 INFO:teuthology.orchestra.run.smithi031.stdout:1098 punch from 0x809242b to 0x809d3ab, (0xaf80 bytes) 2024-09-13T07:12:43.241 INFO:teuthology.orchestra.run.smithi031.stdout:1099 flatten 2024-09-13T07:12:43.241 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:43.241 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:43.242 INFO:teuthology.orchestra.run.smithi031.stdout:1100 write 0x8bcd307 thru 0x8bd7296 (0x9f90 bytes) 2024-09-13T07:12:43.242 INFO:teuthology.orchestra.run.smithi031.stdout:1102 punch from 0x14afb45 to 0x14bfaf6, (0xffb1 bytes) 2024-09-13T07:12:43.242 INFO:teuthology.orchestra.run.smithi031.stdout:1103 write 0x2b12fbf thru 0x2b1aeab (0x7eed bytes) 2024-09-13T07:12:43.242 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:43.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:12:43.243 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:43.244 INFO:teuthology.orchestra.run.smithi031.stdout:1104 trunc from 0x9f2d66a to 0xe8c23d3 2024-09-13T07:12:43.244 INFO:teuthology.orchestra.run.smithi031.stdout:1105 write 0xa87f180 thru 0xa88efa9 (0xfe2a bytes) 2024-09-13T07:12:43.244 INFO:teuthology.orchestra.run.smithi031.stdout:1106 read 0x119df19 thru 0x11a156c (0x3654 bytes) 2024-09-13T07:12:43.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:43.244 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:43.245 INFO:teuthology.orchestra.run.smithi031.stdout:1108 trunc from 0xe8c23d3 to 0x5795b6b 2024-09-13T07:12:43.245 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:43.245 INFO:teuthology.orchestra.run.smithi031.stdout:1109 write 0xdc464be thru 0xdc4ec63 (0x87a6 bytes) 2024-09-13T07:12:43.246 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:12:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:12:43.246 INFO:teuthology.orchestra.run.smithi031.stdout:1110 trunc from 0xdc4ec64 to 0x6dcf41f 2024-09-13T07:12:43.248 INFO:teuthology.orchestra.run.smithi031.stdout:1111 punch from 0x5294bd to 0x52eed2, (0x5a15 bytes) 2024-09-13T07:12:43.248 INFO:teuthology.orchestra.run.smithi031.stdout:1114 read 0xd20a1f thru 0xd30930 (0xff12 bytes) 2024-09-13T07:12:43.248 INFO:teuthology.orchestra.run.smithi031.stdout:1115 read 0x3986bf0 thru 0x398d08b (0x649c bytes) 2024-09-13T07:12:43.249 INFO:teuthology.orchestra.run.smithi031.stdout:1117 read 0x4624dc8 thru 0x46272ed (0x2526 bytes) 2024-09-13T07:12:43.249 INFO:teuthology.orchestra.run.smithi031.stdout:1118 punch from 0x98e999 to 0x999af3, (0xb15a bytes) 2024-09-13T07:12:43.249 INFO:teuthology.orchestra.run.smithi031.stdout:1121 read 0x408ea3d thru 0x4092b54 (0x4118 bytes) 2024-09-13T07:12:43.249 INFO:teuthology.orchestra.run.smithi031.stdout:1122 trunc from 0x6dcf41f to 0xc52a875 2024-09-13T07:12:43.249 INFO:teuthology.orchestra.run.smithi031.stdout:1123 write 0xa1036ab thru 0xa10ee3b (0xb791 bytes) 2024-09-13T07:12:43.249 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:43.239+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:43.250 INFO:teuthology.orchestra.run.smithi031.stdout:1124 punch from 0xd65eb to 0xe176e, (0xb183 bytes) 2024-09-13T07:12:43.250 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.250 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.250 INFO:teuthology.orchestra.run.smithi031.stdout:1127 write 0xd260684 thru 0xd2622da (0x1c57 bytes) 2024-09-13T07:12:43.258 INFO:teuthology.orchestra.run.smithi031.stdout:1128 writesame 0xdc0edf1 thru 0xdc1554c data_size 0x46(0x675c bytes) 2024-09-13T07:12:43.259 INFO:teuthology.orchestra.run.smithi031.stdout:1130 read 0xa2c379e thru 0xa2cf4fd (0xbd60 bytes) 2024-09-13T07:12:43.259 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.273 INFO:teuthology.orchestra.run.smithi031.stdout:1133 trunc from 0xdc1554d to 0x10e6062 2024-09-13T07:12:43.286 INFO:teuthology.orchestra.run.smithi031.stdout:1134 read 0x6e679a thru 0x6f2a25 (0xc28c bytes) 2024-09-13T07:12:43.306 INFO:teuthology.orchestra.run.smithi031.stdout:1135 writesame 0xca37818 thru 0xca417ff data_size 0x56(0x9fe8 bytes) 2024-09-13T07:12:43.306 INFO:teuthology.orchestra.run.smithi031.stdout:1136 write 0x363eb9b thru 0x364dafb (0xef61 bytes) 2024-09-13T07:12:43.316 INFO:teuthology.orchestra.run.smithi031.stdout:1138 trunc from 0xca41800 to 0x9709bc8 2024-09-13T07:12:43.321 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.321 INFO:teuthology.orchestra.run.smithi031.stdout:1141 writesame 0x4c9a918 thru 0x4ca9a2f data_size 0x14(0xf118 bytes) 2024-09-13T07:12:43.328 INFO:teuthology.orchestra.run.smithi031.stdout:1144 read 0x8283d68 thru 0x828a70c (0x69a5 bytes) 2024-09-13T07:12:43.328 INFO:teuthology.orchestra.run.smithi031.stdout:1145 punch from 0x71b5e17 to 0x71c1789, (0xb972 bytes) 2024-09-13T07:12:43.328 INFO:teuthology.orchestra.run.smithi031.stdout:1146 punch from 0x6a86edd to 0x6a923c6, (0xb4e9 bytes) 2024-09-13T07:12:43.337 INFO:teuthology.orchestra.run.smithi031.stdout:1147 write 0xae98713 thru 0xaea7e25 (0xf713 bytes) 2024-09-13T07:12:43.339 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:43.340+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:43.342 INFO:teuthology.orchestra.run.smithi031.stdout:1148 write 0x621a227 thru 0x62230e0 (0x8eba bytes) 2024-09-13T07:12:43.346 INFO:teuthology.orchestra.run.smithi031.stdout:1149 read 0x47b7022 thru 0x47bb63f (0x461e bytes) 2024-09-13T07:12:43.346 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.346 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.346 INFO:teuthology.orchestra.run.smithi031.stdout:1152 read 0x96685b8 thru 0x9670375 (0x7dbe bytes) 2024-09-13T07:12:43.346 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.350 INFO:teuthology.orchestra.run.smithi031.stdout:1154 trunc from 0xaea7e26 to 0xeb17d0a 2024-09-13T07:12:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:1157 read 0xa439de7 thru 0xa43b81c (0x1a36 bytes) 2024-09-13T07:12:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:1158 write 0x147a46f thru 0x14875f5 (0xd187 bytes) 2024-09-13T07:12:43.362 INFO:teuthology.orchestra.run.smithi031.stdout:1160 read 0x4dd8e thru 0x59996 (0xbc09 bytes) 2024-09-13T07:12:43.363 INFO:teuthology.orchestra.run.smithi031.stdout:1162 writesame 0xc2cacdc thru 0xc2cffd6 data_size 0x49(0x52fb bytes) 2024-09-13T07:12:43.367 INFO:teuthology.orchestra.run.smithi031.stdout:1163 write 0x9de274f thru 0x9def6dc (0xcf8e bytes) 2024-09-13T07:12:43.380 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:43.380 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:43.381 INFO:teuthology.orchestra.run.smithi031.stdout:1164 write 0xeb8fdb6 thru 0xeb9565e (0x58a9 bytes) 2024-09-13T07:12:43.390 INFO:teuthology.orchestra.run.smithi031.stdout:1165 writesame 0xbc555af thru 0xbc5f30b data_size 0x7(0x9d5d bytes) 2024-09-13T07:12:43.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:43.395 INFO:teuthology.orchestra.run.smithi031.stdout:1166 writesame 0x95a66c5 thru 0x95aa64d data_size 0x1(0x3f89 bytes) 2024-09-13T07:12:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:1167 read 0x33b6d45 thru 0x33bec43 (0x7eff bytes) 2024-09-13T07:12:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:1169 punch from 0x4373f01 to 0x437ff9a, (0xc099 bytes) 2024-09-13T07:12:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:1170 read 0x73753ef thru 0x7379d5e (0x4970 bytes) 2024-09-13T07:12:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:1171 read 0x888b366 thru 0x8898cf0 (0xd98b bytes) 2024-09-13T07:12:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:1173 write 0xd5902b5 thru 0xd590cbe (0xa0a bytes) 2024-09-13T07:12:43.404 INFO:teuthology.orchestra.run.smithi031.stdout:1174 read 0x79cdef4 thru 0x79d03e8 (0x24f5 bytes) 2024-09-13T07:12:43.419 INFO:teuthology.orchestra.run.smithi031.stdout:1176 trunc from 0xeb9565f to 0x2795ee5 2024-09-13T07:12:43.440 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:43.441+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:43.451 INFO:teuthology.orchestra.run.smithi031.stdout:1177 write 0x7b58042 thru 0x7b59eda (0x1e99 bytes) 2024-09-13T07:12:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:1178 write 0xcc39f83 thru 0xcc48d80 (0xedfe bytes) 2024-09-13T07:12:43.484 INFO:teuthology.orchestra.run.smithi031.stdout:1179 trunc from 0xcc48d81 to 0x61f720f 2024-09-13T07:12:43.495 INFO:teuthology.orchestra.run.smithi031.stdout:1180 write 0x582059f thru 0x582d315 (0xcd77 bytes) 2024-09-13T07:12:43.505 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.505 INFO:teuthology.orchestra.run.smithi031.stdout:1183 read 0x3a1c195 thru 0x3a28fc4 (0xce30 bytes) 2024-09-13T07:12:43.508 INFO:teuthology.orchestra.run.smithi031.stdout:1184 trunc from 0x61f720f to 0x85b01fd 2024-09-13T07:12:43.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.514 INFO:teuthology.orchestra.run.smithi031.stdout:1186 read 0x1de5f3c thru 0x1df41bf (0xe284 bytes) 2024-09-13T07:12:43.514 INFO:teuthology.orchestra.run.smithi031.stdout:1188 punch from 0x802c6d0 to 0x80356a1, (0x8fd1 bytes) 2024-09-13T07:12:43.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.516 INFO:teuthology.orchestra.run.smithi031.stdout:1190 trunc from 0x85b01fd to 0x66f8671 2024-09-13T07:12:43.520 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:43.520 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:43.529 INFO:teuthology.orchestra.run.smithi031.stdout:1191 writesame 0x8007a2d thru 0x800a446 data_size 0x11(0x2a1a bytes) 2024-09-13T07:12:43.529 INFO:teuthology.orchestra.run.smithi031.stdout:1192 write 0x68cb4dd thru 0x68d3053 (0x7b77 bytes) 2024-09-13T07:12:43.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:43.536 INFO:teuthology.orchestra.run.smithi031.stdout:1193 punch from 0x323d364 to 0x3244143, (0x6ddf bytes) 2024-09-13T07:12:43.536 INFO:teuthology.orchestra.run.smithi031.stdout:1196 read 0x64e528c thru 0x64e6bde (0x1953 bytes) 2024-09-13T07:12:43.545 INFO:teuthology.orchestra.run.smithi031.stdout:1198 write 0xb44b905 thru 0xb4586b5 (0xcdb1 bytes) 2024-09-13T07:12:43.546 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:43.547+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:43.554 INFO:teuthology.orchestra.run.smithi031.stdout:1199 read 0xb41f902 thru 0xb42cf6f (0xd66e bytes) 2024-09-13T07:12:43.554 INFO:teuthology.orchestra.run.smithi031.stdout:1200 writesame 0x3def420 thru 0x3dfcdc7 data_size 0x8c(0xd9a8 bytes) 2024-09-13T07:12:43.561 INFO:teuthology.orchestra.run.smithi031.stdout:1201 write 0x673f3e0 thru 0x6749412 (0xa033 bytes) 2024-09-13T07:12:43.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:43.568 INFO:teuthology.orchestra.run.smithi031.stdout:1203 clone 6 order 20 su 65536 sc 10 2024-09-13T07:12:43.642 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:43.643+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:43.646 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:43.646 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:43.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:43.743 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:43.744+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:43.802 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:43.802 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:43.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:43.852 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:43.853+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:43.945 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:43.946+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:43.958 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:43.958 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:43.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:44.051 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:44.052+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:44.114 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:44.114 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:44.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:44.164 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:44.165+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:12:44.256 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:44.257+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:44.278 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:44.278 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:44.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:44.372 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:44.373+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:44.374 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T07:12:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:44.450 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:44.450+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:44.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:44.551 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:12:44.552+0000 7f684b334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33027) UID: 0 2024-09-13T07:12:44.611 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:44.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:1204 write 0xcaac4a3 thru 0xcab2663 (0x61c1 bytes) 2024-09-13T07:12:44.638 INFO:teuthology.orchestra.run.smithi031.stdout:1205 write 0xa7c81c4 thru 0xa7cdc76 (0x5ab3 bytes) 2024-09-13T07:12:44.643 INFO:teuthology.orchestra.run.smithi031.stdout:1206 write 0xab00c3d thru 0xab06e87 (0x624b bytes) 2024-09-13T07:12:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:1207 read 0x310ab09 thru 0x3111f11 (0x7409 bytes) 2024-09-13T07:12:44.646 INFO:teuthology.orchestra.run.smithi031.stdout:1208 write 0x4ede6bd thru 0x4edee3a (0x77e bytes) 2024-09-13T07:12:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:1209 read 0xa074857 thru 0xa07ca05 (0x81af bytes) 2024-09-13T07:12:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:1210 punch from 0x16bef04 to 0x16c506e, (0x616a bytes) 2024-09-13T07:12:44.651 INFO:teuthology.orchestra.run.smithi031.stdout:1212 trunc from 0xcab2664 to 0xda4b129 2024-09-13T07:12:44.651 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:44.652+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:44.656 INFO:teuthology.orchestra.run.smithi031.stdout:1213 read 0x25168fb thru 0x252192c (0xb032 bytes) 2024-09-13T07:12:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:1214 trunc from 0xda4b129 to 0x8b07a1a 2024-09-13T07:12:44.695 INFO:teuthology.orchestra.run.smithi031.stdout:1216 write 0xa8d6a11 thru 0xa8d9de0 (0x33d0 bytes) 2024-09-13T07:12:44.698 INFO:teuthology.orchestra.run.smithi031.stdout:1217 write 0x5f56b03 thru 0x5f64ef7 (0xe3f5 bytes) 2024-09-13T07:12:44.703 INFO:teuthology.orchestra.run.smithi031.stdout:1218 writesame 0x96ba341 thru 0x96c85d0 data_size 0x91(0xe290 bytes) 2024-09-13T07:12:44.711 INFO:teuthology.orchestra.run.smithi031.stdout:1220 trunc from 0xa8d9de1 to 0xe3d2d00 2024-09-13T07:12:44.717 INFO:teuthology.orchestra.run.smithi031.stdout:1221 writesame 0x73a8cf8 thru 0x73b4932 data_size 0x1(0xbc3b bytes) 2024-09-13T07:12:44.734 INFO:teuthology.orchestra.run.smithi031.stdout:1223 write 0x7388bb thru 0x73ca1c (0x4162 bytes) 2024-09-13T07:12:44.746 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:44.747 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:44.752 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:44.753+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:44.753 INFO:teuthology.orchestra.run.smithi031.stdout:1226 trunc from 0xe3d2d00 to 0x2ccc1fd 2024-09-13T07:12:44.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:44.780 INFO:teuthology.orchestra.run.smithi031.stdout:1227 read 0x344a8f thru 0x34af1a (0x648c bytes) 2024-09-13T07:12:44.780 INFO:teuthology.orchestra.run.smithi031.stdout:1229 read 0x201d866 thru 0x20225dd (0x4d78 bytes) 2024-09-13T07:12:44.780 INFO:teuthology.orchestra.run.smithi031.stdout:1233 punch from 0x28806b4 to 0x28817d2, (0x111e bytes) 2024-09-13T07:12:44.780 INFO:teuthology.orchestra.run.smithi031.stdout:1234 write 0xd58842 thru 0xd5ad36 (0x24f5 bytes) 2024-09-13T07:12:44.784 INFO:teuthology.orchestra.run.smithi031.stdout:1236 read 0x286d7c7 thru 0x287c8d5 (0xf10f bytes) 2024-09-13T07:12:44.785 INFO:teuthology.orchestra.run.smithi031.stdout:1237 read 0x8c2f00 thru 0x8cfdbd (0xcebe bytes) 2024-09-13T07:12:44.795 INFO:teuthology.orchestra.run.smithi031.stdout:1238 writesame 0x5e68152 thru 0x5e6ac13 data_size 0xd(0x2ac2 bytes) 2024-09-13T07:12:44.795 INFO:teuthology.orchestra.run.smithi031.stdout:1240 read 0x423ade2 thru 0x4241344 (0x6563 bytes) 2024-09-13T07:12:44.795 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.795 INFO:teuthology.orchestra.run.smithi031.stdout:1243 writesame 0x40d3f58 thru 0x40e3d4e data_size 0x1(0xfdf7 bytes) 2024-09-13T07:12:44.820 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.821 INFO:teuthology.orchestra.run.smithi031.stdout:1245 read 0x49b122b thru 0x49bc480 (0xb256 bytes) 2024-09-13T07:12:44.837 INFO:teuthology.orchestra.run.smithi031.stdout:1246 writesame 0xd94d14e thru 0xd95a3d2 data_size 0x1(0xd285 bytes) 2024-09-13T07:12:44.837 INFO:teuthology.orchestra.run.smithi031.stdout:1248 write 0x24050a1 thru 0x240ae0a (0x5d6a bytes) 2024-09-13T07:12:44.845 INFO:teuthology.orchestra.run.smithi031.stdout:1249 trunc from 0xd95a3d3 to 0x9ad71c6 2024-09-13T07:12:44.851 INFO:teuthology.orchestra.run.smithi031.stdout:1251 read 0x821c2c0 thru 0x821d2e5 (0x1026 bytes) 2024-09-13T07:12:44.851 INFO:teuthology.orchestra.run.smithi031.stdout:1252 read 0x5646d80 thru 0x565444f (0xd6d0 bytes) 2024-09-13T07:12:44.851 INFO:teuthology.orchestra.run.smithi031.stdout:1253 write 0x117364a thru 0x117d90c (0xa2c3 bytes) 2024-09-13T07:12:44.852 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:44.853+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:44.856 INFO:teuthology.orchestra.run.smithi031.stdout:1254 write 0x126c9d thru 0x12d5cf (0x6933 bytes) 2024-09-13T07:12:44.861 INFO:teuthology.orchestra.run.smithi031.stdout:1255 punch from 0x3ee6691 to 0x3ee8479, (0x1de8 bytes) 2024-09-13T07:12:44.861 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.864 INFO:teuthology.orchestra.run.smithi031.stdout:1259 trunc from 0x9ad71c6 to 0x6649e6c 2024-09-13T07:12:44.880 INFO:teuthology.orchestra.run.smithi031.stdout:1260 write 0xa93b3c3 thru 0xa944dd2 (0x9a10 bytes) 2024-09-13T07:12:44.883 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.889 INFO:teuthology.orchestra.run.smithi031.stdout:1262 trunc from 0xa944dd3 to 0x5785a50 2024-09-13T07:12:44.901 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:44.901 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:44.903 INFO:teuthology.orchestra.run.smithi031.stdout:1263 trunc from 0x5785a50 to 0x1398f91 2024-09-13T07:12:44.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:44.928 INFO:teuthology.orchestra.run.smithi031.stdout:1264 punch from 0xf61ac9 to 0xf62105, (0x63c bytes) 2024-09-13T07:12:44.928 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.928 INFO:teuthology.orchestra.run.smithi031.stdout:1266 write 0xb91c2b thru 0xb9901b (0x73f1 bytes) 2024-09-13T07:12:44.935 INFO:teuthology.orchestra.run.smithi031.stdout:1267 write 0x1923b81 thru 0x1927363 (0x37e3 bytes) 2024-09-13T07:12:44.949 INFO:teuthology.orchestra.run.smithi031.stdout:1268 write 0x646e085 thru 0x647a4d2 (0xc44e bytes) 2024-09-13T07:12:44.953 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:44.953+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:44.953 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.958 INFO:teuthology.orchestra.run.smithi031.stdout:1271 trunc from 0x647a4d3 to 0xa071a08 2024-09-13T07:12:44.963 INFO:teuthology.orchestra.run.smithi031.stdout:1274 read 0x59e5213 thru 0x59f0567 (0xb355 bytes) 2024-09-13T07:12:44.963 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.963 INFO:teuthology.orchestra.run.smithi031.stdout:1276 punch from 0x46d51b1 to 0x46d739a, (0x21e9 bytes) 2024-09-13T07:12:44.963 INFO:teuthology.orchestra.run.smithi031.stdout:1278 punch from 0x7525ed0 to 0x753255f, (0xc68f bytes) 2024-09-13T07:12:44.963 INFO:teuthology.orchestra.run.smithi031.stdout:1279 read 0x66ade27 thru 0x66ba374 (0xc54e bytes) 2024-09-13T07:12:44.973 INFO:teuthology.orchestra.run.smithi031.stdout:1280 write 0xe9440eb thru 0xe9480c8 (0x3fde bytes) 2024-09-13T07:12:44.976 INFO:teuthology.orchestra.run.smithi031.stdout:1281 write 0x516d855 thru 0x5176d66 (0x9512 bytes) 2024-09-13T07:12:44.983 INFO:teuthology.orchestra.run.smithi031.stdout:1282 punch from 0x57ec130 to 0x57f6dc0, (0xac90 bytes) 2024-09-13T07:12:44.983 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.983 INFO:teuthology.orchestra.run.smithi031.stdout:1285 writesame 0xb962397 thru 0xb969d7e data_size 0x5e(0x79e8 bytes) 2024-09-13T07:12:44.987 INFO:teuthology.orchestra.run.smithi031.stdout:1286 read 0x98d6570 thru 0x98e26f5 (0xc186 bytes) 2024-09-13T07:12:44.987 INFO:teuthology.orchestra.run.smithi031.stdout:1287 read 0x38cbeb2 thru 0x38d7a7d (0xbbcc bytes) 2024-09-13T07:12:44.987 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:44.997 INFO:teuthology.orchestra.run.smithi031.stdout:1289 trunc from 0xe9480c9 to 0x5b7e95a 2024-09-13T07:12:45.008 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:45.008 INFO:teuthology.orchestra.run.smithi031.stdout:1291 read 0x4f67b9e thru 0x4f6c8ea (0x4d4d bytes) 2024-09-13T07:12:45.008 INFO:teuthology.orchestra.run.smithi031.stdout:1293 writesame 0x3793dc1 thru 0x379bdda data_size 0x13(0x801a bytes) 2024-09-13T07:12:45.024 INFO:teuthology.orchestra.run.smithi031.stdout:1294 writesame 0xc00c64a thru 0xc019853 data_size 0x5f(0xd20a bytes) 2024-09-13T07:12:45.029 INFO:teuthology.orchestra.run.smithi031.stdout:1295 write 0xc2d189f thru 0xc2d8c11 (0x7373 bytes) 2024-09-13T07:12:45.032 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:45.032 INFO:teuthology.orchestra.run.smithi031.stdout:1297 read 0x35af32b thru 0x35be48b (0xf161 bytes) 2024-09-13T07:12:45.033 INFO:teuthology.orchestra.run.smithi031.stdout:1298 read 0x5a4aa35 thru 0x5a4ca26 (0x1ff2 bytes) 2024-09-13T07:12:45.033 INFO:teuthology.orchestra.run.smithi031.stdout:1299 writesame 0xb8bee8e thru 0xb8c9964 data_size 0x1(0xaad7 bytes) 2024-09-13T07:12:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:45.053 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:45.054+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:45.057 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:45.064 INFO:teuthology.orchestra.run.smithi031.stdout:1300 write 0xccc759a thru 0xccd195c (0xa3c3 bytes) 2024-09-13T07:12:45.075 INFO:teuthology.orchestra.run.smithi031.stdout:1301 trunc from 0xccd195d to 0x6268b01 2024-09-13T07:12:45.089 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:45.089 INFO:teuthology.orchestra.run.smithi031.stdout:1303 read 0x21ea61c thru 0x21ee99f (0x4384 bytes) 2024-09-13T07:12:45.089 INFO:teuthology.orchestra.run.smithi031.stdout:1304 read 0x4d0f3fb thru 0x4d12101 (0x2d07 bytes) 2024-09-13T07:12:45.100 INFO:teuthology.orchestra.run.smithi031.stdout:1305 write 0xb45bf43 thru 0xb4642e7 (0x83a5 bytes) 2024-09-13T07:12:45.105 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:45.105 INFO:teuthology.orchestra.run.smithi031.stdout:1309 write 0x4c9ca06 thru 0x4ca5294 (0x888f bytes) 2024-09-13T07:12:45.111 INFO:teuthology.orchestra.run.smithi031.stdout:1310 punch from 0x9d7abf9 to 0x9d85c96, (0xb09d bytes) 2024-09-13T07:12:45.111 INFO:teuthology.orchestra.run.smithi031.stdout:1312 read 0xa12f602 thru 0xa131450 (0x1e4f bytes) 2024-09-13T07:12:45.111 INFO:teuthology.orchestra.run.smithi031.stdout:1313 write 0xa2423ec thru 0xa245ea1 (0x3ab6 bytes) 2024-09-13T07:12:45.114 INFO:teuthology.orchestra.run.smithi031.stdout:1314 read 0x2b7d6aa thru 0x2b8c385 (0xecdc bytes) 2024-09-13T07:12:45.114 INFO:teuthology.orchestra.run.smithi031.stdout:1315 read 0xaa1a39a thru 0xaa206b4 (0x631b bytes) 2024-09-13T07:12:45.114 INFO:teuthology.orchestra.run.smithi031.stdout:1316 read 0x44856ae thru 0x4493261 (0xdbb4 bytes) 2024-09-13T07:12:45.114 INFO:teuthology.orchestra.run.smithi031.stdout:1317 writesame 0x6cf73f4 thru 0x6d055ef data_size 0x12(0xe1fc bytes) 2024-09-13T07:12:45.129 INFO:teuthology.orchestra.run.smithi031.stdout:1318 trunc from 0xb4642e8 to 0x11e7502 2024-09-13T07:12:45.154 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:45.154+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:45.159 INFO:teuthology.orchestra.run.smithi031.stdout:1319 read 0xa26319 thru 0xa271e0 (0xec8 bytes) 2024-09-13T07:12:45.159 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:45.160 INFO:teuthology.orchestra.run.smithi031.stdout:1322 punch from 0x103f3aa to 0x1047375, (0x7fcb bytes) 2024-09-13T07:12:45.167 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:45.167 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:45.176 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:45.178 INFO:teuthology.orchestra.run.smithi031.stdout:1324 writesame 0xc6ce6a4 thru 0xc6d50a6 data_size 0x1(0x6a03 bytes) 2024-09-13T07:12:45.178 INFO:teuthology.orchestra.run.smithi031.stdout:1326 punch from 0xac8452 to 0xaca404, (0x1fb2 bytes) 2024-09-13T07:12:45.179 INFO:teuthology.orchestra.run.smithi031.stdout:1327 write 0x2e1514a thru 0x2e1eccf (0x9b86 bytes) 2024-09-13T07:12:45.186 INFO:teuthology.orchestra.run.smithi031.stdout:1328 trunc from 0xc6d50a7 to 0x741d6d6 2024-09-13T07:12:45.191 INFO:teuthology.orchestra.run.smithi031.stdout:1329 read 0x4ab7a44 thru 0x4ac793a (0xfef7 bytes) 2024-09-13T07:12:45.191 INFO:teuthology.orchestra.run.smithi031.stdout:1330 read 0x2e8cc6a thru 0x2e94e7a (0x8211 bytes) 2024-09-13T07:12:45.194 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:12:45.194 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:12:45.194 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:12:45.194 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T07:12:45.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:12:45.196 INFO:teuthology.orchestra.run.smithi031.stdout:1331 trunc from 0x741d6d6 to 0x399cdd9 2024-09-13T07:12:45.203 INFO:teuthology.orchestra.run.smithi031.stdout:1332 punch from 0x280a3b to 0x28ab23, (0xa0e8 bytes) 2024-09-13T07:12:45.203 INFO:teuthology.orchestra.run.smithi031.stdout:1333 read 0x38c2789 thru 0x38ca625 (0x7e9d bytes) 2024-09-13T07:12:45.214 INFO:teuthology.orchestra.run.smithi031.stdout:1334 trunc from 0x399cdd9 to 0xd48b523 2024-09-13T07:12:45.220 INFO:teuthology.orchestra.run.smithi031.stdout:1335 writesame 0xcf81223 thru 0xcf8615c data_size 0x1(0x4f3a bytes) 2024-09-13T07:12:45.231 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:45.231 INFO:teuthology.orchestra.run.smithi031.stdout:1337 writesame 0x5c38b19 thru 0x5c42a82 data_size 0x37(0x9f6a bytes) 2024-09-13T07:12:45.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:12:45.261 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T07:12:45.262 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2024-09-13T07:12:45.262 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:12:45.262 DEBUG:teuthology.orchestra.run.smithi184:> type ceph-objectstore-tool 2024-09-13T07:12:45.276 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T07:12:45.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-09-13T07:12:45.284 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:45.284 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:45.292 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:45.293+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:45.293 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:45.356 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:45.356+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:45.370 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:45.387 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:45.387 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:45.402 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:45.403 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:45.418 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:45.418 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:45.435 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:45.440 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:45.440 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:45.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:45.440 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:45.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.440 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:914 3.14 3:2ed78eca:::rbd_data.1173f77ecad8.000000000000000a:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.440 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1938257980000007, 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082734499999999999, 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 914 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:45.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246420+0000", 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246424+0000", 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246430+0000", 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8000000000000001e-06 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246434+0000", 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246477+0000", 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2443e-05 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246632+0000", 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015543100000000001 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254694+0000", 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080618979999999996 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.444 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1151 3.9 3:9280c38f:::rbd_data.1173f77ecad8.000000000000010c:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1133857890000005, 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069283519999999996, 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1151 2024-09-13T07:12:45.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326861+0000", 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326860+0000", 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326863+0000", 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326872+0000", 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2199999999999992e-06 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.326886+0000", 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4066999999999999e-05 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327486+0000", 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00060069599999999996 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327560+0000", 2024-09-13T07:12:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3709999999999997e-05 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327669+0000", 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108544 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.330996+0000", 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033276099999999999 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333730+0000", 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002733311 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333767+0000", 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7861000000000002e-05 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.333789+0000", 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1608999999999999e-05 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1153 3.1b e25/19 3:d88532a6:::rbd_data.1173f77ecad8.000000000000015c:head v 25'14, mlcod=25'11)", 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1126789329999998, 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091902019999999997, 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327567+0000", 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327572+0000", 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327579+0000", 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.306e-06 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327585+0000", 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1279999999999996e-06 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327625+0000", 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9227999999999998e-05 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327641+0000", 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6059999999999999e-05 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336708+0000", 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090672949999999995 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336758+0000", 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9870000000000002e-05 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1551 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'89, mlcod=25'86)", 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.6620216299999999, 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077485289999999997, 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778224+0000", 2024-09-13T07:12:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778233+0000", 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4689999999999997e-06 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778245+0000", 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2097e-05 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778252+0000", 2024-09-13T07:12:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.934e-06 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778288+0000", 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6012e-05 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778302+0000", 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4630999999999999e-05 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785938+0000", 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076354329999999996 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785973+0000", 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5663000000000001e-05 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1552 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'90, mlcod=25'86)", 2024-09-13T07:12:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.6613733650000002, 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0071221499999999998, 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778887+0000", 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4409999999999999e-05 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778897+0000", 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0344e-05 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778903+0000", 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7019999999999998e-06 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778933+0000", 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0358e-05 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778955+0000", 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1795999999999999e-05 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785976+0000", 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070212269999999997 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785995+0000", 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8907e-05 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1555 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'93, mlcod=25'86)", 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.6595895430000001, 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0081821140000000007, 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780656+0000", 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780661+0000", 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.079e-06 2024-09-13T07:12:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780670+0000", 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6030000000000005e-06 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780676+0000", 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780720+0000", 2024-09-13T07:12:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3767000000000003e-05 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780739+0000", 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9504000000000002e-05 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788759+0000", 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080197340000000006 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788839+0000", 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0075999999999997e-05 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1556 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'94, mlcod=25'86)", 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.6589127850000001, 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0075812830000000003, 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781333+0000", 2024-09-13T07:12:45.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781337+0000", 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781344+0000", 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781349+0000", 2024-09-13T07:12:45.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781376+0000", 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7826000000000001e-05 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781392+0000", 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5401999999999999e-05 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788842+0000", 2024-09-13T07:12:45.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007450684 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788915+0000", 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2556000000000001e-05 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1557 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'95, mlcod=25'86)", 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.6583215520000003, 2024-09-13T07:12:45.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070583310000000002, 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781928+0000", 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4699999999999998e-06 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781934+0000", 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.302e-06 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781938+0000", 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2559999999999998e-06 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781967+0000", 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9856e-05 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781979+0000", 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1857e-05 2024-09-13T07:12:45.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788917+0000", 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069380750000000001 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788983+0000", 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5906000000000002e-05 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:2859 3.9 3:946400fb:::rbd_data.11734c6039c3.000000000000006d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8225615800000003, 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019805990999999998, 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 2859 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617683+0000", 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617695+0000", 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1626e-05 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617723+0000", 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8252999999999999e-05 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617737+0000", 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3993e-05 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617824+0000", 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6908000000000002e-05 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617958+0000", 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013417 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.621384+0000", 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034260499999999999 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637421+0000", 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016037264999999998 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637430+0000", 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1549999999999996e-06 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637462+0000", 2024-09-13T07:12:45.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1958000000000001e-05 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637491+0000", 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8523000000000001e-05 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3313 3.1a 3:5f0039c6:::rbd_data.11734c6039c3.000000000000003e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.242502752, 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021783180999999999, 2024-09-13T07:12:45.482 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3313 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197742+0000", 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197754+0000", 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2519e-05 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197797+0000", 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3028999999999997e-05 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197803+0000", 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7420000000000003e-06 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197870+0000", 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7123000000000001e-05 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197959+0000", 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8409e-05 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.201633+0000", 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036746299999999999 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219455+0000", 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017821452000000002 2024-09-13T07:12:45.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219464+0000", 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8309999999999996e-06 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219498+0000", 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3948000000000001e-05 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219527+0000", 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8461000000000001e-05 2024-09-13T07:12:45.487 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:3313 3.1a e28/19 ondisk, result = 0)", 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.212445+0000", 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.227801844, 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007012964, 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212445+0000", 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212445+0000", 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212444+0000", 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212448+0000", 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212461+0000", 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2798e-05 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.212469+0000", 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4739999999999999e-06 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219441+0000", 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069722050000000004 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219450+0000", 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2089999999999994e-06 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219458+0000", 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4619999999999999e-06 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.491 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3991 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.676206+0000", 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.7640406410000002, 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070669260000000003, 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3991 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.492 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676206+0000", 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676206+0000", 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676205+0000", 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676208+0000", 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8439999999999998e-06 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676215+0000", 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4389999999999997e-06 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676218+0000", 2024-09-13T07:12:45.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.681655+0000", 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054377269999999998 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.681754+0000", 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8536000000000007e-05 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.682136+0000", 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038192800000000002 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683135+0000", 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00099909400000000007 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683222+0000", 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6916999999999995e-05 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683252+0000", 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0309000000000001e-05 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683273+0000", 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0672000000000001e-05 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.497 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4059 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.7030025640000002, 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010501706, 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4059 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:45.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737245+0000", 2024-09-13T07:12:45.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8199999999999999e-06 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737249+0000", 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5829999999999998e-06 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737255+0000", 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4779999999999999e-06 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.738772+0000", 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001517853 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.738850+0000", 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7206999999999994e-05 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739214+0000", 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036488900000000002 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739761+0000", 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054667599999999995 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747700+0000", 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0079386219999999993 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747726+0000", 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5962999999999999e-05 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747746+0000", 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9899e-05 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4061 3.9 3:92d4a2e4:::rbd_data.11736f46b2f8.000000000000017d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.7002967770000001, 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078848679999999997, 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4061 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739949+0000", 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739954+0000", 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739968+0000", 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3419e-05 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739975+0000", 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3180000000000001e-06 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740015+0000", 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9450999999999999e-05 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9463999999999999e-05 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.741328+0000", 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00125416 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747778+0000", 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064493939999999998 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747786+0000", 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9060000000000002e-06 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747815+0000", 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9634000000000001e-05 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747835+0000", 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9395e-05 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.508 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:4059 3.9 e29/19 ondisk, result = 0)", 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.700172233, 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007677021, 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740077+0000", 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740085+0000", 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3459999999999994e-06 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740091+0000", 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5990000000000001e-06 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747692+0000", 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076006160000000001 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747697+0000", 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1560000000000003e-06 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747751+0000", 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4469999999999999e-05 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4348 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.512 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.4413064690000001, 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072710079999999998, 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4348 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998942+0000", 2024-09-13T07:12:45.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7999999999999999e-06 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998945+0000", 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998947+0000", 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.492e-06 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.001215+0000", 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022681929999999999 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.001296+0000", 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0691000000000004e-05 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.001639+0000", 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034265000000000001 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.002950+0000", 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001310877 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.006161+0000", 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00321193 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.006191+0000", 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9198999999999999e-05 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.006211+0000", 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0356999999999998e-05 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4383 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.4065094359999999, 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077768300000000002, 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.518 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4383 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033739+0000", 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6010000000000002e-06 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033743+0000", 2024-09-13T07:12:45.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9709999999999998e-06 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033746+0000", 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035645+0000", 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001899097 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035686+0000", 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0973000000000002e-05 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035859+0000", 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017274799999999999 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036847+0000", 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00098787000000000007 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041442+0000", 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045950970000000002 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041477+0000", 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5629000000000001e-05 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041514+0000", 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.65e-05 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.523 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4384 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.406439244, 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077768220000000001, 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4384 2024-09-13T07:12:45.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033805+0000", 2024-09-13T07:12:45.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033809+0000", 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1740000000000002e-06 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033814+0000", 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4479999999999996e-06 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033817+0000", 2024-09-13T07:12:45.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.948e-06 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036169+0000", 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023518459999999999 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036222+0000", 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3853000000000001e-05 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036439+0000", 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021612199999999999 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037830+0000", 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013914540000000001 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041522+0000", 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036923220000000001 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041556+0000", 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4044000000000001e-05 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041584+0000", 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7671e-05 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4385 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.4064017830000002, 2024-09-13T07:12:45.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078177509999999995, 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4385 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033844+0000", 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033847+0000", 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033855+0000", 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033859+0000", 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7110000000000001e-06 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036866+0000", 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030072530000000001 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036917+0000", 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0992999999999998e-05 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037097+0000", 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018020300000000001 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.038207+0000", 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001109318 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041592+0000", 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003385047 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041631+0000", 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9196000000000002e-05 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041662+0000", 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1532000000000002e-05 2024-09-13T07:12:45.534 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4386 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.4063366749999999, 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078157709999999991, 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.535 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4386 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033909+0000", 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033912+0000", 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4080000000000001e-06 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033918+0000", 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5169999999999996e-06 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033922+0000", 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1599999999999998e-06 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037379+0000", 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034571020000000001 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037419+0000", 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0751000000000003e-05 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037595+0000", 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017597700000000001 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.038494+0000", 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00089857399999999997 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041667+0000", 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031729610000000002 2024-09-13T07:12:45.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041699+0000", 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2005000000000003e-05 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041726+0000", 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6631e-05 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.541 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:45.541 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:45.542 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:45.543 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:45.543 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:45.544 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:45.543+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:45.643 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:45.644+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:45.647 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:45.648 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:45.648 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:45.648 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:45.658 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:45.744 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:45.745+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:45.805 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:45.805 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:45.806 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:45.806 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:45.806 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:45.814 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:45.844 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:45.846+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:45.919 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:45.919 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:45.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1298 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4576695940000004, 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01046212, 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1298 2024-09-13T07:12:45.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461385+0000", 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461390+0000", 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5210000000000004e-06 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461394+0000", 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2230000000000001e-06 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461397+0000", 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6910000000000002e-06 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464234+0000", 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028368820000000002 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464318+0000", 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4067000000000007e-05 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464664+0000", 2024-09-13T07:12:45.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034640300000000001 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468940+0000", 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042761190000000001 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471764+0000", 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002823739 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471807+0000", 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3485999999999997e-05 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471848+0000", 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0392999999999998e-05 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1300 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.926 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4576364050000006, 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010517047999999999, 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1300 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461418+0000", 2024-09-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461421+0000", 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3489999999999999e-06 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461425+0000", 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8330000000000004e-06 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461427+0000", 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5519999999999999e-06 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465053+0000", 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036258309999999999 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465104+0000", 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0825999999999998e-05 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465312+0000", 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020849900000000001 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468969+0000", 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036562119999999998 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471860+0000", 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002891841 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471903+0000", 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2315999999999999e-05 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471936+0000", 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3132000000000001e-05 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1301 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4576060650000002, 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010564656, 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1301 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461452+0000", 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4690000000000002e-06 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461456+0000", 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0810000000000004e-06 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461459+0000", 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465574+0000", 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041147930000000003 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465655+0000", 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1254999999999994e-05 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466023+0000", 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036823900000000002 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468992+0000", 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029688190000000001 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471945+0000", 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029525609999999998 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471982+0000", 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7837999999999999e-05 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472014+0000", 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1432e-05 2024-09-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1302 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4575758459999992, 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010611844, 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1302 2024-09-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:45.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461481+0000", 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4040000000000002e-06 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461485+0000", 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7969999999999998e-06 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461488+0000", 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6400000000000001e-06 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466476+0000", 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049885140000000003 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466555+0000", 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8208999999999996e-05 2024-09-13T07:12:45.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466936+0000", 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038099899999999998 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469019+0000", 2024-09-13T07:12:45.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020838300000000001 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472021+0000", 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030016019999999999 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472056+0000", 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5295e-05 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472091+0000", 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4876999999999998e-05 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1303 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4575435510000005, 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010655628, 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.945 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1303 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461511+0000", 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461514+0000", 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.418e-06 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461518+0000", 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9219999999999999e-06 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461520+0000", 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.57e-06 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467284+0000", 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057634100000000001 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467336+0000", 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2633999999999998e-05 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467664+0000", 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00032754700000000003 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471710+0000", 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004046231 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472100+0000", 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038953300000000001 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472136+0000", 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6699000000000002e-05 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472167+0000", 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1019999999999998e-05 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.951 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1304 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4575184459999999, 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010707311000000001, 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.953 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1304 2024-09-13T07:12:45.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.953 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:45.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:45.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461536+0000", 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461539+0000", 2024-09-13T07:12:45.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.244e-06 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461543+0000", 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7099999999999998e-06 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461546+0000", 2024-09-13T07:12:45.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5170000000000002e-06 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468072+0000", 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065260650000000002 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468137+0000", 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5036999999999994e-05 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468389+0000", 2024-09-13T07:12:45.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025219400000000002 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471754+0000", 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033649280000000001 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000421183 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472212+0000", 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7153999999999997e-05 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472244+0000", 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1612999999999997e-05 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.959 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1305 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4574872869999993, 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010819396, 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1305 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.960 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461570+0000", 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3860000000000001e-06 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461574+0000", 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461577+0000", 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6340000000000001e-06 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469041+0000", 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0074637030000000004 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469091+0000", 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0197000000000001e-05 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469298+0000", 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020718900000000001 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472253+0000", 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002954702 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472306+0000", 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3198999999999997e-05 2024-09-13T07:12:45.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472351+0000", 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000001e-05 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472387+0000", 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6145e-05 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.965 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1306 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4574566180000001, 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010918835999999999, 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1306 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:45.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461598+0000", 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461602+0000", 2024-09-13T07:12:45.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0860000000000004e-06 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461610+0000", 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0830000000000004e-06 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461616+0000", 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7379999999999999e-06 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469601+0000", 2024-09-13T07:12:45.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0079851690000000003 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469669+0000", 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8316000000000001e-05 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469866+0000", 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000196768 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472259+0000", 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002392786 2024-09-13T07:12:45.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472441+0000", 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018190200000000001 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472485+0000", 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3853999999999997e-05 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472517+0000", 2024-09-13T07:12:45.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2614999999999999e-05 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1307 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4574114510000005, 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0110288, 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.971 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1307 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461643+0000", 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461647+0000", 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9469999999999996e-06 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461656+0000", 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9279999999999999e-06 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461664+0000", 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5989999999999996e-06 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470152+0000", 2024-09-13T07:12:45.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0084883609999999998 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470233+0000", 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1287999999999998e-05 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470632+0000", 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00039872099999999999 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472531+0000", 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018986420000000001 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472593+0000", 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2902000000000002e-05 2024-09-13T07:12:45.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472637+0000", 2024-09-13T07:12:45.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000002e-05 2024-09-13T07:12:45.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472673+0000", 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5839e-05 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1308 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4569678429999993, 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011378667, 2024-09-13T07:12:45.978 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1308 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462086+0000", 2024-09-13T07:12:45.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462092+0000", 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9020000000000004e-06 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462104+0000", 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1807999999999999e-05 2024-09-13T07:12:45.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462113+0000", 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7560000000000005e-06 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470891+0000", 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008778569 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470955+0000", 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3859e-05 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471342+0000", 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038666200000000002 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473305+0000", 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019625630000000001 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473376+0000", 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1354999999999997e-05 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473424+0000", 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8010000000000003e-05 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473466+0000", 2024-09-13T07:12:45.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2132000000000003e-05 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1309 3.1d 3:bcefd1cd:::rbd_data.1173c53667c.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4553094039999994, 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014288309000000001, 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.985 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1309 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463745+0000", 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463751+0000", 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463761+0000", 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3589999999999997e-06 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463767+0000", 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4420000000000003e-06 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472731+0000", 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0089641479999999999 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472858+0000", 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012669100000000001 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473057+0000", 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019875900000000001 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477793+0000", 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047361900000000004 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477983+0000", 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019018299999999999 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478014+0000", 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1062999999999997e-05 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478034+0000", 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.978e-05 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.990 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1373 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.3958289480000001, 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010609936, 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1373 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523231+0000", 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9280000000000001e-06 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523238+0000", 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3509999999999996e-06 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523243+0000", 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0900000000000004e-06 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.527993+0000", 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047498699999999998 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.528077+0000", 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3616999999999996e-05 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.528470+0000", 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000393853 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:45.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532078+0000", 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036080019999999999 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533743+0000", 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001664998 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533790+0000", 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6326000000000003e-05 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533836+0000", 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6490999999999999e-05 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1378 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.3951690800000005, 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010563678, 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:45.997 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1378 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:45.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523889+0000", 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2499999999999998e-06 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523896+0000", 2024-09-13T07:12:45.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523901+0000", 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8679999999999998e-06 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532169+0000", 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082678690000000006 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532251+0000", 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2207999999999996e-05 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532620+0000", 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036913700000000001 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534210+0000", 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015899690000000001 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534377+0000", 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016627800000000001 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534416+0000", 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9626000000000003e-05 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534450+0000", 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3482999999999998e-05 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1379 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.3950427859999994, 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011548345999999999, 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1379 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524016+0000", 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.703e-06 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524026+0000", 2024-09-13T07:12:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524031+0000", 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9190000000000004e-06 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532975+0000", 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008944044 2024-09-13T07:12:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533061+0000", 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5353999999999996e-05 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533391+0000", 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00032971500000000001 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534330+0000", 2024-09-13T07:12:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093939800000000003 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535481+0000", 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011505700000000001 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535522+0000", 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1958e-05 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535561+0000", 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8278999999999997e-05 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1380 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.3948384120000004, 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012748349000000001, 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1380 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524216+0000", 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524221+0000", 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4519999999999996e-06 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524233+0000", 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1652e-05 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524241+0000", 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1100000000000003e-06 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534487+0000", 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010246207 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534570+0000", 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2948999999999995e-05 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534908+0000", 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00033817199999999997 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536427+0000", 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015188420000000001 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536868+0000", 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044121999999999998 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536920+0000", 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1520999999999997e-05 2024-09-13T07:12:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536965+0000", 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5160999999999997e-05 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1555 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1415610170000008, 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011519636, 2024-09-13T07:12:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1555 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777497+0000", 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5609999999999999e-06 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777504+0000", 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5220000000000004e-06 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777509+0000", 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7890000000000002e-06 2024-09-13T07:12:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780260+0000", 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0027512460000000002 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780343+0000", 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3100000000000001e-05 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780598+0000", 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025544100000000002 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786194+0000", 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055953080000000002 2024-09-13T07:12:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788934+0000", 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002739806 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788978+0000", 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4274000000000003e-05 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789014+0000", 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6010999999999998e-05 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1556 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1415410940000008, 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011620324, 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1556 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777517+0000", 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7980000000000001e-06 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777524+0000", 2024-09-13T07:12:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2899999999999997e-06 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777529+0000", 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8319999999999996e-06 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781008+0000", 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034795770000000002 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781090+0000", 2024-09-13T07:12:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1499e-05 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781297+0000", 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000207689 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786431+0000", 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051334140000000002 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789071+0000", 2024-09-13T07:12:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026401670000000001 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789103+0000", 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2400999999999997e-05 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789134+0000", 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0967999999999999e-05 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1557 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1415007349999993, 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011717251999999999, 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1557 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777558+0000", 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2869999999999998e-06 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777568+0000", 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7680000000000003e-06 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777574+0000", 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6520000000000003e-06 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781632+0000", 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040581599999999999 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781679+0000", 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6934000000000002e-05 2024-09-13T07:12:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781867+0000", 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018818900000000001 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786435+0000", 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045683039999999996 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789187+0000", 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002752151 2024-09-13T07:12:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789232+0000", 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.494e-05 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789272+0000", 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9376999999999999e-05 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1558 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1414606099999993, 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011767046999999999, 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1558 2024-09-13T07:12:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777594+0000", 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777597+0000", 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2140000000000001e-06 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777605+0000", 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777610+0000", 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782276+0000", 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046664000000000002 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782321+0000", 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4489e-05 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782507+0000", 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018627699999999999 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786438+0000", 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039313780000000001 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789315+0000", 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002876871 2024-09-13T07:12:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789342+0000", 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6415e-05 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789362+0000", 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9868999999999999e-05 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1559 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.1413713370000007, 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011750661000000001, 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1559 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777683+0000", 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777690+0000", 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8730000000000001e-06 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777703+0000", 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3016999999999999e-05 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777710+0000", 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782759+0000", 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050484909999999996 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782845+0000", 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6082999999999996e-05 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783061+0000", 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000215732 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788822+0000", 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057617689999999999 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789389+0000", 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00056687899999999995 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789416+0000", 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7132999999999999e-05 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789435+0000", 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8199999999999999e-05 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.048 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:46.050 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:46.050 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:46.051+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:46.151 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:46.152+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:46.196 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:46.196 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:46.196 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:46.196 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:46.205 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:46.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:12:46.289 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:46.290+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:46.348 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:46.348 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:46.349 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:46.349 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:46.349 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:46.353 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:46.354+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:46.357 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:46.416 INFO:teuthology.orchestra.run.smithi184.stdout:3.f 2024-09-13T07:12:46.416 INFO:teuthology.orchestra.run.smithi184.stdout:3.e 2024-09-13T07:12:46.416 INFO:teuthology.orchestra.run.smithi184.stdout:3.d 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:3.7 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:3.16 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:3.15 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:3.13 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:3.12 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:3.11 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:3.14 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:2.0 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:1.2 2024-09-13T07:12:46.417 INFO:teuthology.orchestra.run.smithi184.stdout:1.0 2024-09-13T07:12:46.454 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:46.454+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:46.501 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:46.501 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4257.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 25.907339370999999, 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036526549999999999, 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:46.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594197+0000", 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594211+0000", 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.399e-05 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594220+0000", 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2359999999999993e-06 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594231+0000", 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0293e-05 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594280+0000", 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9261000000000002e-05 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594301+0000", 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0744999999999999e-05 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597812+0000", 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035110950000000001 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597850+0000", 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:908 3.10 3:09fecc17:::rbd_data.1173f77ecad8.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2572889610000004, 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040361349999999997, 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.506 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 908 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244247+0000", 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244253+0000", 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6389999999999997e-06 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244264+0000", 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1544e-05 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244270+0000", 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244309+0000", 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8838999999999999e-05 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244385+0000", 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6097e-05 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244591+0000", 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000206749 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246079+0000", 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014876679999999999 2024-09-13T07:12:46.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248230+0000", 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021511780000000001 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248264+0000", 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3283e-05 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248284+0000", 2024-09-13T07:12:46.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0598000000000002e-05 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:913 3.1e e25/19 3:7c3d5185:::rbd_data.1173f77ecad8.0000000000000005:head v 25'4, mlcod=24'2)", 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.254930731, 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004381462, 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246612+0000", 2024-09-13T07:12:46.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9499999999999998e-06 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246622+0000", 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0217000000000001e-05 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246627+0000", 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0540000000000002e-06 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5330000000000002e-05 2024-09-13T07:12:46.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246684+0000", 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1645e-05 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.250954+0000", 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042697819999999997 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.250988+0000", 2024-09-13T07:12:46.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4215999999999999e-05 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:922 3.a 3:52856a69:::rbd_data.1173f77ecad8.000000000000000f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2501652149999991, 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058310970000000004, 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 922 2024-09-13T07:12:46.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251371+0000", 2024-09-13T07:12:46.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251376+0000", 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9450000000000001e-06 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251387+0000", 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0786e-05 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251392+0000", 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3310000000000003e-06 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251426+0000", 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3767999999999999e-05 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251503+0000", 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7410000000000006e-05 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.257203+0000", 2024-09-13T07:12:46.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056996900000000003 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1010 3.10 3:0d2ccec7:::rbd_data.1173f77ecad8.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2323117929999992, 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042914140000000003, 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1010 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269224+0000", 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269230+0000", 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7030000000000003e-06 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269243+0000", 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3215e-05 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269248+0000", 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0889999999999999e-06 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269292+0000", 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3928999999999999e-05 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269370+0000", 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8163000000000004e-05 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269581+0000", 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021031700000000001 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.270174+0000", 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059290199999999997 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273429+0000", 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00325524 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273485+0000", 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5926e-05 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273517+0000", 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1826000000000003e-05 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1052 3.10 3:0982ea78:::rbd_data.1173f77ecad8.000000000000003e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2193249930000007, 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052248729999999997, 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1052 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282211+0000", 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282217+0000", 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6350000000000003e-06 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282228+0000", 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1323e-05 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282245+0000", 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6807e-05 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282279+0000", 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4100000000000002e-05 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282365+0000", 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6012000000000003e-05 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287437+0000", 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050720519999999996 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1061 3.10 3:0c912c88:::rbd_data.1173f77ecad8.0000000000000044:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.216414447, 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005098455, 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1061 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.530 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285122+0000", 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285126+0000", 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3340000000000003e-06 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285135+0000", 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9039999999999998e-06 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285142+0000", 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285181+0000", 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8717000000000002e-05 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285304+0000", 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012349400000000001 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290221+0000", 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049169610000000001 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1067 3.10 3:0982ea78:::rbd_data.1173f77ecad8.000000000000003e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2139510530000006, 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035721809999999998, 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.534 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1067 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:46.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287585+0000", 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287590+0000", 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2560000000000004e-06 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287599+0000", 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3349999999999996e-06 2024-09-13T07:12:46.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287604+0000", 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5470000000000004e-06 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287646+0000", 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1992e-05 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287714+0000", 2024-09-13T07:12:46.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7594000000000004e-05 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.288113+0000", 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00039889400000000001 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.288598+0000", 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000484981 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291095+0000", 2024-09-13T07:12:46.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024973759999999999 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291137+0000", 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1987999999999998e-05 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291158+0000", 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0919000000000001e-05 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1073 3.10 3:09473a54:::rbd_data.1173f77ecad8.0000000000000049:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2111582900000002, 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004417575, 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1073 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290378+0000", 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290383+0000", 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3179999999999997e-06 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290396+0000", 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2492e-05 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290402+0000", 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.759e-06 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290439+0000", 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6915000000000001e-05 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290530+0000", 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0853999999999994e-05 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.294796+0000", 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042662339999999998 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1074 3.10 3:0c912c88:::rbd_data.1173f77ecad8.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2108516609999995, 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055866580000000004, 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1074 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:46.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290691+0000", 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0800000000000002e-06 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290699+0000", 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7390000000000005e-06 2024-09-13T07:12:46.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290704+0000", 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7280000000000003e-06 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290731+0000", 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7299000000000001e-05 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290776+0000", 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4848000000000002e-05 2024-09-13T07:12:46.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290902+0000", 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012560100000000001 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.294803+0000", 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039015579999999998 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296179+0000", 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013752879999999999 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296230+0000", 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1270999999999998e-05 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296272+0000", 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2035e-05 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1125 3.a 3:5135e61f:::rbd_data.1173f77ecad8.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.549 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.1900206680000007, 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00420258, 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1125 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311520+0000", 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8919999999999999e-06 2024-09-13T07:12:46.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311531+0000", 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0805000000000001e-05 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311538+0000", 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1400000000000002e-06 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311577+0000", 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8395000000000001e-05 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311664+0000", 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7609999999999996e-05 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311792+0000", 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012732499999999999 2024-09-13T07:12:46.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315178+0000", 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033866970000000001 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315667+0000", 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00048881399999999996 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315698+0000", 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0321999999999999e-05 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315719+0000", 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1369000000000002e-05 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1130 3.1e e25/19 3:7b48162d:::rbd_data.1173f77ecad8.000000000000005c:head v 25'13, mlcod=25'10)", 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.1887856600000006, 2024-09-13T07:12:46.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005273247, 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312757+0000", 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9410000000000004e-06 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312769+0000", 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2799999999999999e-05 2024-09-13T07:12:46.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312776+0000", 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8199999999999999e-06 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312819+0000", 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2293999999999999e-05 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312838+0000", 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9123e-05 2024-09-13T07:12:46.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.317986+0000", 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051480340000000001 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.318025+0000", 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8914e-05 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1320 3.10 3:0e76b587:::rbd_data.1173c53667c.0000000000000057:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 497989] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.559 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.472830+0000", 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.0287068690000005, 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033972469999999999, 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1320 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472830+0000", 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472830+0000", 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472828+0000", 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472837+0000", 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9609999999999994e-06 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472856+0000", 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8556e-05 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472865+0000", 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9590000000000001e-06 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472924+0000", 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8947999999999999e-05 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473047+0000", 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012370999999999999 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473239+0000", 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000191375 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.475070+0000", 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018309540000000001 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.476166+0000", 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010964289999999999 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.476205+0000", 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.879e-05 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.476227+0000", 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2402999999999999e-05 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1348 3.10 3:0e76b587:::rbd_data.1173c53667c.0000000000000057:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9844989300000009, 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038489119999999999, 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1348 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517036+0000", 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517046+0000", 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7370000000000001e-06 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517066+0000", 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9658999999999999e-05 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517076+0000", 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0217999999999999e-05 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517134+0000", 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7779999999999999e-05 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517218+0000", 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4738000000000001e-05 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517335+0000", 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116573 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.519614+0000", 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022788830000000002 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520800+0000", 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001186063 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520846+0000", 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6161000000000001e-05 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520887+0000", 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0960000000000001e-05 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1536 3.a 3:5448ec73:::rbd_data.1173c53667c.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7360198679999996, 2024-09-13T07:12:46.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004075932, 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1536 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765516+0000", 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765521+0000", 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0499999999999999e-06 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765532+0000", 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1138000000000001e-05 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765538+0000", 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765575+0000", 2024-09-13T07:12:46.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7283000000000002e-05 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765630+0000", 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4818999999999998e-05 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765695+0000", 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5215000000000005e-05 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766354+0000", 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00065942400000000003 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769502+0000", 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031472549999999998 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769551+0000", 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9577999999999999e-05 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769593+0000", 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1983000000000001e-05 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1542 3.a 3:52be2a25:::rbd_data.1173c53667c.0000000000000043:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.7318341269999991, 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005091641, 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.576 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1542 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769702+0000", 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769708+0000", 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3509999999999998e-06 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769722+0000", 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4334e-05 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769730+0000", 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7330000000000003e-06 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769789+0000", 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9077999999999999e-05 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769860+0000", 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0832000000000002e-05 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769978+0000", 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118257 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.770500+0000", 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052145399999999997 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774714+0000", 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042146079999999999 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774760+0000", 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.583e-05 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774795+0000", 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4288000000000001e-05 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4643447159999994, 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.582 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:12:46.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:12:46.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:46.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1768 3.a e25/19 ondisk, result = 0)", 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:46.587 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.4494716309999998, 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010131984, 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052064+0000", 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052072+0000", 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9529999999999999e-06 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052090+0000", 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8263000000000001e-05 2024-09-13T07:12:46.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052102+0000", 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2078e-05 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062047+0000", 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0099446580000000003 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062059+0000", 2024-09-13T07:12:46.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2455e-05 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062197+0000", 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013813200000000001 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3345 3.10 3:0be6394f:::rbd_data.11734c6039c3.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2154235309999999, 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035746290000000002, 2024-09-13T07:12:46.591 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3345 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286111+0000", 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999971 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286123+0000", 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2471e-05 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286146+0000", 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3136999999999999e-05 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286156+0000", 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4550000000000001e-06 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286216+0000", 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0245999999999999e-05 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286223+0000", 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1590000000000004e-06 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288195+0000", 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001972279 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288274+0000", 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8882999999999997e-05 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288352+0000", 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8158999999999995e-05 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289021+0000", 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00066851300000000003 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289615+0000", 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059398999999999997 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289657+0000", 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1989000000000001e-05 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289688+0000", 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1257000000000002e-05 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4259 3.10 3:0a0f6fac:::rbd_data.11736f46b2f8.0000000000000082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:46.597 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.938427+0000", 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.563109549, 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035730979999999998, 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4259 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938427+0000", 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938427+0000", 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938425+0000", 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938436+0000", 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0910999999999999e-05 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938456+0000", 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0041e-05 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938464+0000", 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6660000000000008e-06 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938508+0000", 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4554000000000001e-05 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938561+0000", 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2615999999999999e-05 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938620+0000", 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9178999999999997e-05 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.939277+0000", 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00065729600000000003 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.941923+0000", 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026458879999999999 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.941966+0000", 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-05 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.942001+0000", 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.455e-05 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.603 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:46.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:46.603 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:46.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:46.605 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:46.605+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:46.705 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:46.706+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:46.711 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:46.711 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:46.711 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:46.711 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:46.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:46.806 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:46.807+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:46.867 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:46.867 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:46.867 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:46.867 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:46.867 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:46.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:46.920 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:46.921+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:46.932 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op export --pgid 3.16 --file /var/log/ceph/exp.3.16.7 2024-09-13T07:12:46.985 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:46.985 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:46.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:46.985 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:46.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:906 3.1 3:80f3f876:::rbd_data.1173f77ecad8.0000000000000009:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.242359+0000", 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7425816189999992, 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061750620000000003, 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 906 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242359+0000", 2024-09-13T07:12:46.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242359+0000", 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:46.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242358+0000", 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242364+0000", 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4389999999999997e-06 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243101+0000", 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073671900000000005 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243111+0000", 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7909999999999999e-06 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243405+0000", 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029383000000000002 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243526+0000", 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121458 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248534+0000", 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050079510000000001 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:916 3.12 3:4ade7e99:::rbd_data.1173f77ecad8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7382787099999994, 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081196749999999998, 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 916 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:46.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246661+0000", 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246666+0000", 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4450000000000004e-06 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246681+0000", 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5104e-05 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246689+0000", 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246739+0000", 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9855000000000003e-05 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246823+0000", 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3820999999999996e-05 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.247040+0000", 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021744099999999999 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.247583+0000", 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054293699999999996 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254695+0000", 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071119429999999999 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254741+0000", 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5507999999999999e-05 2024-09-13T07:12:46.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254782+0000", 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1047999999999997e-05 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:920 3.1 3:87900178:::rbd_data.1173f77ecad8.000000000000000e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7364112939999998, 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060296539999999997, 2024-09-13T07:12:46.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 920 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248534+0000", 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248544+0000", 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7210000000000006e-06 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248550+0000", 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0589999999999999e-06 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248587+0000", 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6783999999999999e-05 2024-09-13T07:12:46.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248704+0000", 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117199 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254559+0000", 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058553570000000003 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:921 3.1 3:80f3f876:::rbd_data.1173f77ecad8.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.249040+0000", 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7359011970000005, 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005632448, 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 921 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249040+0000", 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249040+0000", 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249039+0000", 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249045+0000", 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249059+0000", 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3925e-05 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249066+0000", 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:12:47.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249114+0000", 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8628000000000003e-05 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249198+0000", 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3560000000000006e-05 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249396+0000", 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019819600000000001 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249786+0000", 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039016700000000002 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254602+0000", 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004815204 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254639+0000", 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7107999999999998e-05 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254672+0000", 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3246e-05 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:969 3.17 3:e9f80bb1:::rbd_data.1173f77ecad8.000000000000001e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.262845+0000", 2024-09-13T07:12:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7220960650000006, 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055654340000000002, 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 969 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.262845+0000", 2024-09-13T07:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.262845+0000", 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.262843+0000", 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.262849+0000", 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9869999999999996e-06 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.262860+0000", 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0203e-05 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.262867+0000", 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.262930+0000", 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3751999999999995e-05 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.263080+0000", 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000149493 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.268410+0000", 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053302030000000004 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1015 3.6 3:65878748:::rbd_data.1173f77ecad8.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.270768+0000", 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7141724039999993, 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059243009999999999, 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1015 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270768+0000", 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270768+0000", 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270767+0000", 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270773+0000", 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270783+0000", 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9240000000000002e-06 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270789+0000", 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:12:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270831+0000", 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1709000000000003e-05 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270927+0000", 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6464000000000003e-05 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271210+0000", 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028254799999999998 2024-09-13T07:12:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276509+0000", 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005298929 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276607+0000", 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8502e-05 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276656+0000", 2024-09-13T07:12:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8566000000000002e-05 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276693+0000", 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6808999999999999e-05 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1107 3.12 3:4dcd22ad:::rbd_data.1173f77ecad8.0000000000000056:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.303224+0000", 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.681717184, 2024-09-13T07:12:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055617690000000003, 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1107 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303224+0000", 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303224+0000", 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303223+0000", 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303227+0000", 2024-09-13T07:12:47.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000002e-06 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303238+0000", 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0509999999999999e-05 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303244+0000", 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1040000000000003e-06 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303285+0000", 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1451e-05 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303361+0000", 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5564999999999999e-05 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303509+0000", 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147825 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.306254+0000", 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027456680000000002 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308685+0000", 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024306729999999999 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308750+0000", 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4671000000000005e-05 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308785+0000", 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5602999999999998e-05 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.3 epoch 25/19 rep_tid 30 entries 25'11 (0'0) error 3:c2b5a8ac:::rbd_data.1173f77ecad8.0000000000000061:head by client.4467.0:1128 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'8)", 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.312215+0000", 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6727261519999992, 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056486449999999999, 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312215+0000", 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312215+0000", 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312214+0000", 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312221+0000", 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.973e-06 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312234+0000", 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3406e-05 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312242+0000", 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312314+0000", 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2157999999999996e-05 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.317863+0000", 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055494949999999998 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1153 3.1b 3:d88532a6:::rbd_data.1173f77ecad8.000000000000015c:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6580454210000006, 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010064101000000001, 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1153 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326898+0000", 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2660000000000002e-06 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326906+0000", 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4789999999999999e-06 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326910+0000", 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326959+0000", 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8327e-05 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327025+0000", 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6161999999999994e-05 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327352+0000", 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032693799999999998 2024-09-13T07:12:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.330855+0000", 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003503501 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.336879+0000", 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060237880000000004 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.336918+0000", 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9400999999999998e-05 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.336959+0000", 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0972e-05 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1317 3.17 3:ec069b17:::rbd_data.1173c53667c.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5127655830000002, 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057872000000000002, 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1317 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472174+0000", 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472180+0000", 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2859999999999996e-06 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472193+0000", 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2581e-05 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472200+0000", 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5349999999999999e-06 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472255+0000", 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4957999999999998e-05 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472395+0000", 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014002400000000001 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472584+0000", 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018895399999999999 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475965+0000", 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033809349999999998 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.477910+0000", 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019448849999999999 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.477941+0000", 2024-09-13T07:12:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0874999999999998e-05 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.477962+0000", 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1268999999999999e-05 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1344 3.6 3:65d64014:::rbd_data.1173c53667c.0000000000000053:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4707705709999992, 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064892170000000002, 2024-09-13T07:12:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1344 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514174+0000", 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514182+0000", 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0930000000000003e-06 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514189+0000", 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514715+0000", 2024-09-13T07:12:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052648699999999996 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.515279+0000", 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056402199999999996 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.515416+0000", 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013629000000000001 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.518278+0000", 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028619800000000001 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.520605+0000", 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023270019999999999 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.520639+0000", 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3970000000000002e-05 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.520659+0000", 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0684000000000001e-05 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1631 3.17 3:efa3965d:::rbd_data.1173c53667c.00000000000000aa:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.867573+0000", 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1173681210000002, 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020209943000000001, 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1631 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867573+0000", 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867573+0000", 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867571+0000", 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867578+0000", 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6379999999999998e-06 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867590+0000", 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2086000000000001e-05 2024-09-13T07:12:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867596+0000", 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867656+0000", 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0504999999999998e-05 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867764+0000", 2024-09-13T07:12:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107629 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.871357+0000", 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003592947 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887667+0000", 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016309939999999998 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887716+0000", 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9199000000000001e-05 2024-09-13T07:12:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887754+0000", 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8519000000000002e-05 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887783+0000", 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8160000000000001e-05 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:1631 3.17 e25/19 ondisk, result = 0)", 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1034342269999993, 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006282429, 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881510+0000", 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4649999999999996e-06 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881523+0000", 2024-09-13T07:12:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2604e-05 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881529+0000", 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5520000000000001e-06 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887704+0000", 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061750750000000004 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887711+0000", 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4980000000000003e-06 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887789+0000", 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8014000000000002e-05 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1640 3.1 3:87b03873:::rbd_data.1173c53667c.00000000000000a0:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.0877302869999994, 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015780835999999999, 2024-09-13T07:12:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1640 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897214+0000", 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.617e-06 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897225+0000", 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0747000000000001e-05 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897230+0000", 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9960000000000001e-06 2024-09-13T07:12:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897280+0000", 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9514000000000001e-05 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897413+0000", 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133304 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.899854+0000", 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024406950000000001 2024-09-13T07:12:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912923+0000", 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013069173999999999 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912931+0000", 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6950000000000006e-06 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912965+0000", 2024-09-13T07:12:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.396e-05 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912991+0000", 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5856000000000001e-05 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:1640 3.1 e25/19 ondisk, result = 0)", 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.907414+0000", 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.0775264169999996, 2024-09-13T07:12:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005510989, 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.907414+0000", 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.907414+0000", 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.907414+0000", 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.907418+0000", 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0879999999999997e-06 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.907430+0000", 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2033e-05 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.907436+0000", 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912912+0000", 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054759930000000002 2024-09-13T07:12:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912919+0000", 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1810000000000004e-06 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912925+0000", 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1828 3.18 e26/19 3:1c28841c:::rbd_data.1173c53667c.00000000000000b8:head v 26'17, mlcod=25'15)", 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.8359054149999992, 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057524409999999996, 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149056+0000", 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1333000000000001e-05 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149065+0000", 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9160000000000007e-06 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149074+0000", 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.541e-06 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149115+0000", 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1014999999999999e-05 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149135+0000", 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0435999999999999e-05 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154761+0000", 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056255630000000001 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154788+0000", 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.688e-05 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:2858 3.1b 3:d93b9695:::rbd_data.11734c6039c3.000000000000006c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.617151+0000", 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.3677896399999998, 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081434639999999996, 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2858 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617151+0000", 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617151+0000", 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617149+0000", 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617160+0000", 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.128e-05 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617178+0000", 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.73e-05 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617184+0000", 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8260000000000004e-06 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617233+0000", 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8573999999999999e-05 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617366+0000", 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000132884 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.618603+0000", 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012372139999999999 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625232+0000", 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066293020000000001 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625240+0000", 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2860000000000002e-06 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625270+0000", 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0134999999999999e-05 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625295+0000", 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4683999999999999e-05 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3319 3.17 3:ebf18aab:::rbd_data.11734c6039c3.0000000000000030:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.228186+0000", 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7567549400000004, 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086840040000000004, 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3319 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228186+0000", 2024-09-13T07:12:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228186+0000", 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228184+0000", 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228193+0000", 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1649999999999995e-06 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228210+0000", 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7116999999999999e-05 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228217+0000", 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9999999999999999e-06 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228260+0000", 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2342000000000002e-05 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228266+0000", 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0059999999999998e-06 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.229955+0000", 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001689308 2024-09-13T07:12:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.230025+0000", 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9962999999999994e-05 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.231187+0000", 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011621730000000001 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236776+0000", 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055889290000000003 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236820+0000", 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3544000000000002e-05 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236851+0000", 2024-09-13T07:12:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1420000000000001e-05 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236870+0000", 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8689999999999999e-05 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4054 3.6 3:60dce801:::rbd_data.11736f46b2f8.000000000000011a:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.727538+0000", 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2574031980000004, 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080239769999999998, 2024-09-13T07:12:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4054 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727538+0000", 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727538+0000", 2024-09-13T07:12:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727536+0000", 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727544+0000", 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3710000000000002e-06 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727561+0000", 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6979000000000001e-05 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727569+0000", 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4479999999999994e-06 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727621+0000", 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1413999999999999e-05 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727723+0000", 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102645 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.729014+0000", 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012910549999999999 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735413+0000", 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063984469999999998 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735460+0000", 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7068999999999999e-05 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735519+0000", 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9487999999999997e-05 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735561+0000", 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2222000000000003e-05 2024-09-13T07:12:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4076 3.12 3:4f45d645:::rbd_data.11736f46b2f8.000000000000018e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.761011+0000", 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2239296509999997, 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056740929999999998, 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4076 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761011+0000", 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761011+0000", 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761010+0000", 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761015+0000", 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6920000000000001e-06 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761026+0000", 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1003e-05 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761031+0000", 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4729999999999999e-06 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761065+0000", 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4010000000000001e-05 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761112+0000", 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6999000000000002e-05 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761172+0000", 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9312999999999999e-05 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761581+0000", 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000409116 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.766635+0000", 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050541370000000002 2024-09-13T07:12:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.766662+0000", 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7069000000000001e-05 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.766685+0000", 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3045999999999999e-05 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.089 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:47.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:47.091 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:47.092+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:47.191 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:47.192+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:12:47.225 INFO:teuthology.orchestra.run.smithi031.stdout:1339 clone 7 order 22 su 1048576 sc 9 2024-09-13T07:12:47.243 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:47.243 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:47.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:47.292 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:47.293+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:47.394 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:47.395+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:47.398 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:47.398 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:47.399 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:47.399 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:47.399 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:47.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:47.495 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:47.496+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:47.515 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4257.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.931497177000001, 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01398422, 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4257", 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/806497074", 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584054+0000", 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584061+0000", 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1929999999999998e-06 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584069+0000", 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8240000000000006e-06 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584077+0000", 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584133+0000", 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5896000000000002e-05 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584145+0000", 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2293000000000001e-05 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593792+0000", 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096463009999999995 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593914+0000", 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121994 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.594023+0000", 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109152 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597204+0000", 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003181021 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597968+0000", 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076444399999999997 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597998+0000", 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0051000000000001e-05 2024-09-13T07:12:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.598040+0000", 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1548999999999998e-05 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.605365275, 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044056909999999998, 2024-09-13T07:12:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910187+0000", 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910194+0000", 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1439999999999997e-06 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910205+0000", 2024-09-13T07:12:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1198e-05 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910210+0000", 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9590000000000003e-06 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910267+0000", 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7473999999999999e-05 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910291+0000", 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3456999999999999e-05 2024-09-13T07:12:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914549+0000", 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042583059999999999 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914593+0000", 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3954e-05 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.604960886000001, 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040318009999999998, 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910592+0000", 2024-09-13T07:12:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910591+0000", 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910597+0000", 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8200000000000002e-06 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910608+0000", 2024-09-13T07:12:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1592e-05 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.910613+0000", 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913555+0000", 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029417639999999999 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913596+0000", 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0697e-05 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914598+0000", 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001001907 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914624+0000", 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5865000000000001e-05 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.604481364000002, 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039444399999999996, 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911071+0000", 2024-09-13T07:12:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911070+0000", 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911098+0000", 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7339999999999999e-05 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911112+0000", 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.385e-05 2024-09-13T07:12:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.911117+0000", 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9030000000000003e-06 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913900+0000", 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002783761 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.913923+0000", 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2313000000000001e-05 2024-09-13T07:12:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.914975+0000", 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010525319999999999 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:21.915016+0000", 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0593999999999998e-05 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'67)", 2024-09-13T07:12:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.423640890000001, 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054341140000000003, 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091912+0000", 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091911+0000", 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091969+0000", 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7450000000000001e-05 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091976+0000", 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.091979+0000", 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2399999999999999e-06 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092008+0000", 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9040999999999999e-05 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092022+0000", 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3229e-05 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097327+0000", 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053053470000000002 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097346+0000", 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9045000000000001e-05 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:113 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'67)", 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.422994729999999, 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048006739999999996, 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092558+0000", 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092557+0000", 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092562+0000", 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092572+0000", 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8530000000000004e-06 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092576+0000", 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4229999999999998e-06 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092608+0000", 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1735e-05 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.092621+0000", 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3268000000000001e-05 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097347+0000", 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047255589999999998 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097359+0000", 2024-09-13T07:12:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1678999999999999e-05 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:114 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'76, mlcod=16'71)", 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.422355528000001, 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041827970000000003, 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093197+0000", 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093201+0000", 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8770000000000001e-06 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093211+0000", 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:12:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093216+0000", 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093240+0000", 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4624e-05 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.093252+0000", 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1391e-05 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097360+0000", 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041079139999999998 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.097380+0000", 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0392999999999999e-05 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:117 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'79, mlcod=16'71)", 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.421182535, 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058774140000000001, 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094370+0000", 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094374+0000", 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0720000000000001e-06 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094385+0000", 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.146e-05 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094390+0000", 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-06 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094417+0000", 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7350000000000001e-05 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.094427+0000", 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0257e-05 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.100222+0000", 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057946849999999999 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.100248+0000", 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5565e-05 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:141 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'95)", 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.997597078999998, 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052230660000000002, 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517956+0000", 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517955+0000", 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517961+0000", 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6149999999999996e-06 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517968+0000", 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.875e-06 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.517973+0000", 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4150000000000001e-06 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519906+0000", 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001932879 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.519919+0000", 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2877999999999999e-05 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.523151+0000", 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032328880000000002 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.523179+0000", 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7209999999999999e-05 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:144 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.990086347999998, 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052055089999999997, 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525466+0000", 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525530+0000", 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4370999999999997e-05 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525539+0000", 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0510000000000002e-06 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525543+0000", 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9310000000000001e-06 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525575+0000", 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1359000000000002e-05 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525586+0000", 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530640+0000", 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050537189999999999 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530672+0000", 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1887999999999997e-05 2024-09-13T07:12:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4210.0:145 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.989599160000001, 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047501109999999996, 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525954+0000", 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.525953+0000", 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526013+0000", 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0170000000000002e-05 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526021+0000", 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5390000000000002e-06 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526024+0000", 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2049999999999998e-06 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526057+0000", 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.345e-05 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.526070+0000", 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2315e-05 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530675+0000", 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004605217 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:22.530704+0000", 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.866e-05 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1017 3.19 3:9dd4f9cc:::rbd_data.1173f77ecad8.000000000000002a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.270928+0000", 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.244624476, 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055151430000000001, 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1017 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270928+0000", 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270928+0000", 2024-09-13T07:12:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270927+0000", 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270934+0000", 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.776e-06 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270949+0000", 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.517e-05 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270957+0000", 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3159999999999997e-06 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271003+0000", 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6372000000000003e-05 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271088+0000", 2024-09-13T07:12:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4435000000000001e-05 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271304+0000", 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021626100000000001 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273154+0000", 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018497749999999999 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276312+0000", 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003158498 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276402+0000", 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9903999999999998e-05 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276443+0000", 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0991000000000001e-05 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1020 3.8 e25/19 3:11bc453b:::rbd_data.1173f77ecad8.0000000000000029:head v 25'4, mlcod=23'2)", 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271717+0000", 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.243835533, 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043542770000000001, 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271717+0000", 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271717+0000", 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271716+0000", 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271724+0000", 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0029999999999994e-06 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271738+0000", 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3152999999999999e-05 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271745+0000", 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0700000000000001e-06 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271875+0000", 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130295 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271903+0000", 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7756999999999999e-05 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276021+0000", 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004118498 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276071+0000", 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0257999999999999e-05 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1015 3.6 e25/19 3:65878748:::rbd_data.1173f77ecad8.0000000000000026:head v 25'5, mlcod=25'3)", 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.271731+0000", 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.243821261000001, 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004286397, 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271731+0000", 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271731+0000", 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271731+0000", 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271738+0000", 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.063e-06 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271749+0000", 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0937e-05 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271756+0000", 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9330000000000004e-06 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271812+0000", 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6056e-05 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271824+0000", 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2489e-05 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.275978+0000", 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041543159999999999 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276018+0000", 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9425000000000003e-05 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.8 epoch 25/19 rep_tid 31 entries 25'5 (0'0) error 3:133f0075:::rbd_data.1173f77ecad8.0000000000000031:head by client.4467.0:1028 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 23'2)", 2024-09-13T07:12:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.273984+0000", 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.241568439, 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047682369999999998, 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273984+0000", 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273984+0000", 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273984+0000", 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.273991+0000", 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9049999999999997e-06 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.274005+0000", 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.274012+0000", 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1749999999999999e-06 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.274059+0000", 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6737000000000003e-05 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.278752+0000", 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046938479999999996 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1097 3.0 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299078+0000", 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.216474791, 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005452307, 2024-09-13T07:12:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1097 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299078+0000", 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299078+0000", 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299077+0000", 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299093+0000", 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6507999999999998e-05 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299106+0000", 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2889e-05 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299113+0000", 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1790000000000002e-06 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299167+0000", 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4036999999999998e-05 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299219+0000", 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1854999999999997e-05 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299331+0000", 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112402 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304350+0000", 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050189750000000002 2024-09-13T07:12:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304429+0000", 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8455999999999996e-05 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304486+0000", 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7584000000000003e-05 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304530+0000", 2024-09-13T07:12:47.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3711000000000002e-05 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.6 epoch 25/19 rep_tid 72 entries 25'10 (0'0) error 3:64cbcb8d:::rbd_data.1173f77ecad8.0000000000000059:head by client.4467.0:1105 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'8)", 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302561+0000", 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.212991822999999, 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325689999999999, 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302561+0000", 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302561+0000", 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302560+0000", 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302569+0000", 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302589+0000", 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9576999999999999e-05 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302601+0000", 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2692e-05 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302665+0000", 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3868000000000006e-05 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.306993+0000", 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043283540000000004 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1776 3.4 3:2614c9b5:::rbd_data.1173c53667c.00000000000000b2:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:38.075527+0000", 2024-09-13T07:12:47.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4400253440000004, 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047552239999999997, 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1776 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075527+0000", 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075527+0000", 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075526+0000", 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075532+0000", 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075543+0000", 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0961999999999999e-05 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075548+0000", 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075586+0000", 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7426999999999999e-05 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.075678+0000", 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2022999999999996e-05 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.076499+0000", 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000821454 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.080162+0000", 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036632240000000001 2024-09-13T07:12:47.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.080176+0000", 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4022000000000001e-05 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.080232+0000", 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5067e-05 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.080283+0000", 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1047000000000001e-05 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3066 3.4 3:27ec9434:::rbd_object_map.11734c6039c3:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.883623+0000", 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.6319294620000004, 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040094659999999997, 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3066 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883623+0000", 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883623+0000", 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883621+0000", 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T07:12:47.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883635+0000", 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4007e-05 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883659+0000", 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4009e-05 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883668+0000", 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3139999999999993e-06 2024-09-13T07:12:47.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883728+0000", 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9716999999999998e-05 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.883843+0000", 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114675 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884155+0000", 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031290099999999998 2024-09-13T07:12:47.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887474+0000", 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033183380000000001 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887579+0000", 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105166 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887609+0000", 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9709e-05 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887633+0000", 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4049999999999998e-05 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3283 3.19 3:9d5c61ce:::rbd_data.11734c6039c3.000000000000006a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.133186+0000", 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.3823669990000003, 2024-09-13T07:12:47.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045617009999999996, 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3283 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.133186+0000", 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.133186+0000", 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.133185+0000", 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.133189+0000", 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4520000000000002e-06 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.133197+0000", 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999992e-06 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.133202+0000", 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6389999999999997e-06 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.133834+0000", 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063189400000000003 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:47.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.134199+0000", 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036515499999999998 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.134314+0000", 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011458199999999999 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.135517+0000", 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012029370000000001 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:47.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.137702+0000", 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021845810000000001 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.137729+0000", 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7064e-05 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.137747+0000", 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8808e-05 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:47.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:47.610 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:47.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:47.611 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:47.612+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:47.711 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:47.712+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:47.759 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:47.759 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:47.759 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:47.759 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:47.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:47.852 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:47.853+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:47.913 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:47.914+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:47.914 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:47.914 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:47.914 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:47.914 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:47.914 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:47.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:48.014 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:48.015+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:48.060 INFO:teuthology.orchestra.run.smithi184.stderr:Exporting 3.16 info 3.16( v 30'52 (0'0,30'52] local-lis/les=19/20 n=10 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-13T07:12:48.067 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6847a25e:::rbd_data.11734b163e95.0000000000000013:head# 2024-09-13T07:12:48.069 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:69bcfbd0:::rbd_data.1173f77ecad8.0000000000000063:head# 2024-09-13T07:12:48.069 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:69d6b5f1:::rbd_data.1173f77ecad8.000000000000005b:head# 2024-09-13T07:12:48.070 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6a67f01d:::rbd_data.1173f77ecad8.0000000000000027:head# 2024-09-13T07:12:48.070 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6c86404b:::rbd_data.1173c53667c.0000000000000006:head# 2024-09-13T07:12:48.070 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6c95c7f1:::rbd_data.11736f46b2f8.000000000000016a:head# 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.773606672, 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:48.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.772128647000001, 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:48.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:12:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.769311598, 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:12:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1319 3.0 e25/19 3:06065967:::rbd_data.1173c53667c.0000000000000056:head v 25'7, mlcod=25'5)", 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.598626203, 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026907649999999999, 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472901+0000", 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472909+0000", 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472925+0000", 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6019e-05 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472933+0000", 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8089999999999993e-06 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472979+0000", 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6034e-05 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.473001+0000", 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2218999999999999e-05 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475552+0000", 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025503309999999999 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475593+0000", 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1468999999999999e-05 2024-09-13T07:12:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1998826380000001, 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.187259042, 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3179 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'130, mlcod=27'127)", 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.0439764120000001, 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027322449999999999, 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027551+0000", 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027558+0000", 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027569+0000", 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.059e-05 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027575+0000", 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5420000000000002e-06 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027617+0000", 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1328999999999997e-05 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027637+0000", 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0407999999999998e-05 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030260+0000", 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026228509999999998 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030284+0000", 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4389999999999999e-05 2024-09-13T07:12:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3786 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'28, mlcod=29'24)", 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.555104+0000", 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5164242249999997, 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002407991, 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555104+0000", 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555104+0000", 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555104+0000", 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555107+0000", 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7579999999999998e-06 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555115+0000", 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555127+0000", 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2252000000000001e-05 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555154+0000", 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6579000000000001e-05 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.555166+0000", 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2444e-05 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.557471+0000", 2024-09-13T07:12:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002304932 2024-09-13T07:12:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.557512+0000", 2024-09-13T07:12:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1270000000000003e-05 2024-09-13T07:12:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3869 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'38, mlcod=29'35)", 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.587716+0000", 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4838122010000001, 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025663579999999999, 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587716+0000", 2024-09-13T07:12:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587716+0000", 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587716+0000", 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587729+0000", 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529999999999999e-05 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587738+0000", 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8319999999999995e-06 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587744+0000", 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587770+0000", 2024-09-13T07:12:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6548999999999999e-05 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.587784+0000", 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3072e-05 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.590246+0000", 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024620710000000001 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.590283+0000", 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6984999999999999e-05 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4436772739999997, 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:12:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:12:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:12:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:12:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:12:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:12:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:12:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:12:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.443140047, 2024-09-13T07:12:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:12:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:12:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3914 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'57, mlcod=29'53)", 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.629005+0000", 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.442523778, 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002332443, 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629005+0000", 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629005+0000", 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629004+0000", 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629008+0000", 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.002e-06 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629018+0000", 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3349999999999996e-06 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629023+0000", 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1309999999999997e-06 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629081+0000", 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8671999999999997e-05 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.629095+0000", 2024-09-13T07:12:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3552e-05 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631326+0000", 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022307630000000002 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631337+0000", 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.146e-05 2024-09-13T07:12:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3040584109999998, 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:12:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:12:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:12:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:12:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:12:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:12:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:12:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:12:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4162 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'113, mlcod=29'110)", 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2572506929999996, 2024-09-13T07:12:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028573499999999998, 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814277+0000", 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814283+0000", 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814293+0000", 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0299000000000001e-05 2024-09-13T07:12:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814298+0000", 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814345+0000", 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7707999999999998e-05 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814365+0000", 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9836000000000001e-05 2024-09-13T07:12:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817100+0000", 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027346969999999999 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817135+0000", 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5194999999999998e-05 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2567111180000001, 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.229227013, 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4195 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'122, mlcod=29'119)", 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2236914060000004, 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027697870000000001, 2024-09-13T07:12:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847836+0000", 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847841+0000", 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847850+0000", 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2390000000000002e-06 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847854+0000", 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847880+0000", 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847900+0000", 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0015000000000001e-05 2024-09-13T07:12:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850568+0000", 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002668283 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850607+0000", 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8349999999999997e-05 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2232125890000001, 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1715202949999997, 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:12:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4252 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'140, mlcod=29'136)", 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.919744+0000", 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1517844019999997, 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026713800000000001, 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919744+0000", 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919744+0000", 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919743+0000", 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919749+0000", 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919761+0000", 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1259e-05 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919767+0000", 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.438e-06 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919793+0000", 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5675999999999999e-05 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.919805+0000", 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2269000000000001e-05 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.922381+0000", 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025758959999999998 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.922415+0000", 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4530000000000003e-05 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.166 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:48.167 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6d931999:::rbd_data.1173f77ecad8.000000000000001c:head# 2024-09-13T07:12:48.167 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6db07b17:::rbd_data.1173f77ecad8.0000000000000023:head# 2024-09-13T07:12:48.167 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6dcc6547:::rbd_data.1173f77ecad8.0000000000000035:head# 2024-09-13T07:12:48.167 INFO:teuthology.orchestra.run.smithi184.stderr:Read #3:6f3c8750:::rbd_data.1173f77ecad8.000000000000002f:head# 2024-09-13T07:12:48.167 INFO:teuthology.orchestra.run.smithi184.stderr:Export successful 2024-09-13T07:12:48.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:48.169 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:48.170+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:48.269 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:48.270+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:48.278 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone6 from 0xd48b523 (overlap 0x11e7502) to 0xc8851b 2024-09-13T07:12:48.331 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:48.331 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:48.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:48.345 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T07:12:48.370 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:48.371+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:48.466 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:48.466 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:48.471 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:48.473+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:48.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:48.550 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.550 INFO:teuthology.orchestra.run.smithi031.stdout:1341 read 0x3ad5aa4 thru 0x3ad8390 (0x28ed bytes) 2024-09-13T07:12:48.560 INFO:teuthology.orchestra.run.smithi031.stdout:1342 trunc from 0xd48b523 to 0x50b5d03 2024-09-13T07:12:48.572 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:48.573+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:48.579 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.579 INFO:teuthology.orchestra.run.smithi031.stdout:1347 read 0x30bdda3 thru 0x30cd7e3 (0xfa41 bytes) 2024-09-13T07:12:48.580 INFO:teuthology.orchestra.run.smithi031.stdout:1348 trunc from 0x50b5d03 to 0x42694c2 2024-09-13T07:12:48.591 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --force --op remove --pgid 3.16 2024-09-13T07:12:48.597 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:923 3.7 3:e0d0441f:::rbd_data.1173f77ecad8.0000000000000010:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.345740341000001, 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054522629999999997, 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 923 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251785+0000", 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251784+0000", 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251790+0000", 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8300000000000001e-06 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251802+0000", 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1935e-05 2024-09-13T07:12:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251807+0000", 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.251857+0000", 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9515000000000003e-05 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.252009+0000", 2024-09-13T07:12:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015179700000000001 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.257238+0000", 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052290870000000003 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1075 3.b 3:d41379e8:::rbd_data.1173f77ecad8.0000000000000041:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.307064883000001, 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057904439999999996, 2024-09-13T07:12:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290461+0000", 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290460+0000", 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290466+0000", 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:12:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290477+0000", 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1595e-05 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290485+0000", 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7360000000000001e-06 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290527+0000", 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1690000000000002e-05 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290610+0000", 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3321999999999999e-05 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.290800+0000", 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019058399999999999 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.293341+0000", 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025410139999999999 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296160+0000", 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028181460000000001 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296209+0000", 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9055999999999999e-05 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.296251+0000", 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2703000000000001e-05 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1098 3.15 e25/19 3:afeba560:::rbd_data.1173f77ecad8.000000000000004f:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.298063361000001, 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091051389999999999, 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6639999999999998e-06 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299484+0000", 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4609e-05 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299491+0000", 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.584e-06 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299534+0000", 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640999999999998e-05 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299553+0000", 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8564999999999999e-05 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308532+0000", 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089789510000000006 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308568+0000", 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5896000000000003e-05 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.c epoch 25/19 rep_tid 48 entries 25'5 (0'0) error 3:32647bc6:::rbd_data.1173f77ecad8.0000000000000057:head by client.4467.0:1100 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'2)", 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.296265376000001, 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073257039999999997, 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301266+0000", 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5239999999999997e-06 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301279+0000", 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2856e-05 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301296+0000", 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6858e-05 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301369+0000", 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2631000000000003e-05 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308586+0000", 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072174989999999996 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1104 3.1e 3:7e73e8c7:::rbd_data.1173f77ecad8.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.295663916000001, 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009729985, 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1104 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301861+0000", 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301866+0000", 2024-09-13T07:12:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301879+0000", 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3083e-05 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301893+0000", 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4656e-05 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301950+0000", 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6671e-05 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302027+0000", 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6859000000000004e-05 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302260+0000", 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023318300000000001 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303132+0000", 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000871487 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311492+0000", 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083604839999999996 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311550+0000", 2024-09-13T07:12:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7701000000000002e-05 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311592+0000", 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2024000000000003e-05 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1106 3.c e25/19 3:30d6dbcc:::rbd_data.1173f77ecad8.0000000000000050:head v 25'6, mlcod=25'3)", 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.295084119, 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090851709999999995, 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302441+0000", 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302447+0000", 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7969999999999997e-06 2024-09-13T07:12:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302460+0000", 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2772000000000001e-05 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302466+0000", 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302507+0000", 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1014999999999999e-05 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302520+0000", 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2706e-05 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311485+0000", 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089648929999999998 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311527+0000", 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2259000000000003e-05 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1118 3.1e 3:7d8c2ee8:::rbd_data.1173f77ecad8.000000000000005f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.288506729, 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056840500000000004, 2024-09-13T07:12:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1118 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309019+0000", 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309018+0000", 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309023+0000", 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0950000000000003e-06 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309034+0000", 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0857e-05 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309041+0000", 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0639999999999996e-06 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309094+0000", 2024-09-13T07:12:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2936000000000003e-05 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.309241+0000", 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146618 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.314703+0000", 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054623320000000003 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1130 3.1e 3:7b48162d:::rbd_data.1173f77ecad8.000000000000005c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.285314538, 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060295670000000004, 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1130 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312211+0000", 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312210+0000", 2024-09-13T07:12:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312217+0000", 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.939e-06 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312229+0000", 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2269000000000001e-05 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312237+0000", 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8310000000000001e-06 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312306+0000", 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8515000000000004e-05 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312412+0000", 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312643+0000", 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023097699999999999 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.317531+0000", 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048874529999999999 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318137+0000", 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000606352 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318196+0000", 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8816000000000001e-05 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.318241+0000", 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5170999999999998e-05 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1154 3.b 3:d3009503:::rbd_data.1173f77ecad8.0000000000000168:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.269870020000001, 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059705729999999999, 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1154 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327656+0000", 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327655+0000", 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327661+0000", 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6860000000000003e-06 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327672+0000", 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1809e-05 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327680+0000", 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9400000000000002e-06 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327729+0000", 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9023000000000001e-05 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327807+0000", 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7768999999999994e-05 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327961+0000", 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000153616 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.330853+0000", 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002891897 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333519+0000", 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026667359999999998 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333576+0000", 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6786000000000001e-05 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333626+0000", 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0144e-05 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1155 3.b 3:d723b608:::rbd_data.1173f77ecad8.00000000000001d6:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.26938451, 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058401670000000003, 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1155 2024-09-13T07:12:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328141+0000", 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328145+0000", 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5820000000000003e-06 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328156+0000", 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1255e-05 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328163+0000", 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0149999999999998e-06 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328502+0000", 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033881500000000001 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328582+0000", 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166000000000005e-05 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.328777+0000", 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019505799999999999 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333240+0000", 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044629070000000003 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333900+0000", 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065926699999999995 2024-09-13T07:12:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333943+0000", 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3027e-05 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.333981+0000", 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8828999999999997e-05 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1828 3.18 3:1c28841c:::rbd_data.1173c53667c.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write 963576~17461] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.451128114999999, 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085613430000000008, 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1828 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146396+0000", 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146417+0000", 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1087999999999999e-05 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146431+0000", 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3788000000000001e-05 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146442+0000", 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0545e-05 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146535+0000", 2024-09-13T07:12:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3067999999999997e-05 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146542+0000", 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5290000000000003e-06 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148549+0000", 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002006407 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148675+0000", 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126549 2024-09-13T07:12:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148818+0000", 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014280900000000001 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.152082+0000", 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032638709999999998 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154899+0000", 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028167520000000001 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154935+0000", 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6392e-05 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154959+0000", 2024-09-13T07:12:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4170000000000001e-05 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3311 3.18 3:1ef9c6ed:::rbd_data.11734c6039c3.00000000000000c8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.4284061680000004, 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02606054, 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3311 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169119+0000", 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169124+0000", 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169134+0000", 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0325e-05 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169139+0000", 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.916e-06 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169183+0000", 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4369999999999997e-05 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169319+0000", 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013582199999999999 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.173537+0000", 2024-09-13T07:12:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042175870000000001 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195024+0000", 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021487336999999999 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195119+0000", 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4963000000000005e-05 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195157+0000", 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7743000000000001e-05 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195180+0000", 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3272e-05 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:3311 3.18 e27/19 ondisk, result = 0)", 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.4118317170000001, 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093332329999999998, 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185693+0000", 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185700+0000", 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999996e-06 2024-09-13T07:12:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185720+0000", 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0055e-05 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185730+0000", 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9709999999999999e-06 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195010+0000", 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092796179999999999 2024-09-13T07:12:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195018+0000", 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1999999999999994e-06 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195027+0000", 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3335 3.b 3:d2887617:::rbd_data.11734c6039c3.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.3423753710000002, 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018824244, 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3335 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255155+0000", 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9250000000000003e-06 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255165+0000", 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0682e-05 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255172+0000", 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1600000000000001e-06 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255212+0000", 2024-09-13T07:12:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9329999999999998e-05 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255260+0000", 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7981e-05 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.258002+0000", 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027423500000000002 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273857+0000", 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015854844999999999 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273892+0000", 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4938999999999999e-05 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273927+0000", 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5515999999999997e-05 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273975+0000", 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7307e-05 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:3335 3.b e28/19 ondisk, result = 0)", 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.268127+0000", 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.3293988240000001, 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058552220000000002, 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.268127+0000", 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.268127+0000", 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.268126+0000", 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.268130+0000", 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9149999999999998e-06 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.268143+0000", 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2269000000000001e-05 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.268151+0000", 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0900000000000005e-06 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273875+0000", 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057246220000000004 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273888+0000", 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273982+0000", 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4542000000000004e-05 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4246 3.1e 3:78aceff8:::rbd_data.11736f46b2f8.00000000000000c8:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.6971042379999997, 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017416903000000001, 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4246 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900421+0000", 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900426+0000", 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900437+0000", 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1062e-05 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900442+0000", 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900480+0000", 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8012000000000001e-05 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900590+0000", 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109262 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902585+0000", 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019956090000000002 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917754+0000", 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015168683000000001 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917788+0000", 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4190000000000003e-05 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917817+0000", 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8486000000000001e-05 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917838+0000", 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1911000000000001e-05 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4247 3.7 3:e146183c:::rbd_data.11736f46b2f8.00000000000000c9:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.6970748630000001, 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070352820000000003, 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4247 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900453+0000", 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900457+0000", 2024-09-13T07:12:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9369999999999997e-06 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900460+0000", 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.937e-06 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900512+0000", 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2119e-05 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900622+0000", 2024-09-13T07:12:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011024800000000001 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902136+0000", 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001513723 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907373+0000", 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052366859999999999 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907409+0000", 2024-09-13T07:12:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.693e-05 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907453+0000", 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3262e-05 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907486+0000", 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3541999999999998e-05 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4249 3.1e 3:7927132b:::rbd_data.11736f46b2f8.00000000000000cf:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.6929639280000002, 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013320103, 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4249 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904560+0000", 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904568+0000", 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9489999999999996e-06 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904582+0000", 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3619e-05 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904589+0000", 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4660000000000002e-06 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.910641+0000", 2024-09-13T07:12:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060512719999999999 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.910732+0000", 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1306999999999998e-05 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.911772+0000", 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001039653 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917783+0000", 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060117579999999999 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917844+0000", 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0378999999999999e-05 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917866+0000", 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1858999999999999e-05 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917882+0000", 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6317999999999999e-05 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:4246 3.1e e29/19 ondisk, result = 0)", 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.6895927070000001, 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098236480000000008, 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907932+0000", 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907936+0000", 2024-09-13T07:12:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9820000000000002e-06 2024-09-13T07:12:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907945+0000", 2024-09-13T07:12:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8179999999999993e-06 2024-09-13T07:12:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907950+0000", 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6500000000000004e-06 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917744+0000", 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097943720000000008 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917749+0000", 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8099999999999997e-06 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917757+0000", 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4271 3.1e 3:7927132b:::rbd_data.11736f46b2f8.00000000000000cf:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.940513+0000", 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.6570130760000001, 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057501510000000002, 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4271 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940513+0000", 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940513+0000", 2024-09-13T07:12:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940512+0000", 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940519+0000", 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6810000000000001e-06 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940529+0000", 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0716999999999999e-05 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940537+0000", 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3989999999999998e-06 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940573+0000", 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6220000000000002e-05 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940618+0000", 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4749000000000001e-05 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.940698+0000", 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0610000000000002e-05 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.941536+0000", 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083769400000000005 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.946191+0000", 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046550970000000004 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.946236+0000", 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5019999999999999e-05 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.946263+0000", 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6689000000000001e-05 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:48.704 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:48.706 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:48.707 INFO:teuthology.orchestra.run.smithi031.stdout:1350 writesame 0xd12f1b8 thru 0xd13e3cb data_size 0x2(0xf214 bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1351 write 0xd13393 thru 0xd136b4 (0x322 bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1353 write 0x1129fd7 thru 0x112b75a (0x1784 bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1354 write 0x3e3b944 thru 0x3e3d1c2 (0x187f bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1356 read 0xbaaabde thru 0xbab7789 (0xcbac bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1357 write 0xb0484b3 thru 0xb04d79e (0x52ec bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1358 read 0xbcfa201 thru 0xbcfd5fd (0x33fd bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1359 write 0x76c135b thru 0x76cf10c (0xddb2 bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1360 read 0x21b8805 thru 0x21c2940 (0xa13c bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1367 trunc from 0xd13e3cc to 0x97b7ef6 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1369 trunc from 0x97b7ef6 to 0x6cc7008 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1370 read 0x21e556c thru 0x21ebc88 (0x671d bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1371 punch from 0x13667f7 to 0x13756fb, (0xef04 bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1373 write 0x41251d8 thru 0x4134303 (0xf12c bytes) 2024-09-13T07:12:48.708 INFO:teuthology.orchestra.run.smithi031.stdout:1374 read 0x403bd04 thru 0x403cacc (0xdc9 bytes) 2024-09-13T07:12:48.709 INFO:teuthology.orchestra.run.smithi031.stdout:1375 punch from 0xfa2c0d to 0xfa449a, (0x188d bytes) 2024-09-13T07:12:48.709 INFO:teuthology.orchestra.run.smithi031.stdout:1376 write 0xa912ff2 thru 0xa915038 (0x2047 bytes) 2024-09-13T07:12:48.709 INFO:teuthology.orchestra.run.smithi031.stdout:1377 trunc from 0xa915039 to 0xd71e6f 2024-09-13T07:12:48.709 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:48.706+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:48.713 INFO:teuthology.orchestra.run.smithi031.stdout:1378 punch from 0xac378d to 0xad1870, (0xe0e3 bytes) 2024-09-13T07:12:48.713 INFO:teuthology.orchestra.run.smithi031.stdout:1379 punch from 0xc90a6f to 0xc92bb6, (0x2147 bytes) 2024-09-13T07:12:48.713 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.725 INFO:teuthology.orchestra.run.smithi031.stdout:1381 write 0x67485ee thru 0x675445f (0xbe72 bytes) 2024-09-13T07:12:48.727 INFO:teuthology.orchestra.run.smithi031.stdout:1382 read 0x14ecc4d thru 0x14f9cb1 (0xd065 bytes) 2024-09-13T07:12:48.736 INFO:teuthology.orchestra.run.smithi031.stdout:1383 writesame 0x8c17337 thru 0x8c240d4 data_size 0x5d(0xcd9e bytes) 2024-09-13T07:12:48.736 INFO:teuthology.orchestra.run.smithi031.stdout:1385 writesame 0x504045d thru 0x50485ce data_size 0x3f(0x8172 bytes) 2024-09-13T07:12:48.749 INFO:teuthology.orchestra.run.smithi031.stdout:1387 write 0xca3ad8e thru 0xca3cbf4 (0x1e67 bytes) 2024-09-13T07:12:48.752 INFO:teuthology.orchestra.run.smithi031.stdout:1388 writesame 0x18307f1 thru 0x183fc8e data_size 0x93(0xf49e bytes) 2024-09-13T07:12:48.755 INFO:teuthology.orchestra.run.smithi031.stdout:1389 trunc from 0xca3cbf5 to 0xe1727fa 2024-09-13T07:12:48.761 INFO:teuthology.orchestra.run.smithi031.stdout:1390 read 0x6c7e12e thru 0x6c86462 (0x8335 bytes) 2024-09-13T07:12:48.761 INFO:teuthology.orchestra.run.smithi031.stdout:1391 writesame 0xb7e0fcc thru 0xb7ee377 data_size 0x4c(0xd3ac bytes) 2024-09-13T07:12:48.763 INFO:teuthology.orchestra.run.smithi031.stdout:1392 writesame 0x2bb0948 thru 0x2bb6ec3 data_size 0x1e(0x657c bytes) 2024-09-13T07:12:48.766 INFO:teuthology.orchestra.run.smithi031.stdout:1393 punch from 0x831f9b5 to 0x83272ea, (0x7935 bytes) 2024-09-13T07:12:48.777 INFO:teuthology.orchestra.run.smithi031.stdout:1394 trunc from 0xe1727fa to 0x48a5f19 2024-09-13T07:12:48.789 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.804 INFO:teuthology.orchestra.run.smithi031.stdout:1396 write 0xcaeaa46 thru 0xcaf82e2 (0xd89d bytes) 2024-09-13T07:12:48.806 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:48.807+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:48.808 INFO:teuthology.orchestra.run.smithi031.stdout:1397 read 0x24a159c thru 0x24b0cee (0xf753 bytes) 2024-09-13T07:12:48.808 INFO:teuthology.orchestra.run.smithi031.stdout:1398 punch from 0x8e145f5 to 0x8e18e17, (0x4822 bytes) 2024-09-13T07:12:48.808 INFO:teuthology.orchestra.run.smithi031.stdout:1399 write 0x849042d thru 0x84a009a (0xfc6e bytes) 2024-09-13T07:12:48.811 INFO:teuthology.orchestra.run.smithi031.stdout:1400 punch from 0x2e182fe to 0x2e25bcd, (0xd8cf bytes) 2024-09-13T07:12:48.811 INFO:teuthology.orchestra.run.smithi031.stdout:1401 write 0xa9a8e2 thru 0xa9f1aa (0x48c9 bytes) 2024-09-13T07:12:48.812 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.812 INFO:teuthology.orchestra.run.smithi031.stdout:1405 write 0x621af89 thru 0x622825c (0xd2d4 bytes) 2024-09-13T07:12:48.815 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.815 INFO:teuthology.orchestra.run.smithi031.stdout:1407 read 0x2d15497 thru 0x2d1ec86 (0x97f0 bytes) 2024-09-13T07:12:48.815 INFO:teuthology.orchestra.run.smithi031.stdout:1408 read 0xbf53aef thru 0xbf5f9a5 (0xbeb7 bytes) 2024-09-13T07:12:48.815 INFO:teuthology.orchestra.run.smithi031.stdout:1410 read 0x2afaba3 thru 0x2b08b70 (0xdfce bytes) 2024-09-13T07:12:48.815 INFO:teuthology.orchestra.run.smithi031.stdout:1411 writesame 0x42eefee thru 0x42efe6b data_size 0xe(0xe7e bytes) 2024-09-13T07:12:48.817 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.817 INFO:teuthology.orchestra.run.smithi031.stdout:1414 write 0x2af5bfd thru 0x2afdecb (0x82cf bytes) 2024-09-13T07:12:48.821 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:48.821 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:48.821 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:48.821 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:48.827 INFO:teuthology.orchestra.run.smithi031.stdout:1415 write 0xe643742 thru 0xe643f91 (0x850 bytes) 2024-09-13T07:12:48.830 INFO:teuthology.orchestra.run.smithi031.stdout:1416 write 0xd3881fc thru 0xd39204a (0x9e4f bytes) 2024-09-13T07:12:48.830 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:48.831 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:48.831 INFO:teuthology.orchestra.run.smithi031.stdout:1420 punch from 0x5def60a to 0x5def851, (0x247 bytes) 2024-09-13T07:12:48.831 INFO:teuthology.orchestra.run.smithi031.stdout:1424 writesame 0x7ab3b3b thru 0x7ac30ef data_size 0x3(0xf5b5 bytes) 2024-09-13T07:12:48.841 INFO:teuthology.orchestra.run.smithi031.stdout:1426 punch from 0x9609366 to 0x960b802, (0x249c bytes) 2024-09-13T07:12:48.842 INFO:teuthology.orchestra.run.smithi031.stdout:1427 punch from 0x5b8abe to 0x5bb738, (0x2c7a bytes) 2024-09-13T07:12:48.842 INFO:teuthology.orchestra.run.smithi031.stdout:1428 punch from 0x5ed125d to 0x5ed7954, (0x66f7 bytes) 2024-09-13T07:12:48.842 INFO:teuthology.orchestra.run.smithi031.stdout:1431 write 0x750ce79 thru 0x7518366 (0xb4ee bytes) 2024-09-13T07:12:48.844 INFO:teuthology.orchestra.run.smithi031.stdout:1432 read 0xa5846af thru 0xa5896ec (0x503e bytes) 2024-09-13T07:12:48.845 INFO:teuthology.orchestra.run.smithi031.stdout:1433 write 0x182760a thru 0x1835856 (0xe24d bytes) 2024-09-13T07:12:48.846 INFO:teuthology.orchestra.run.smithi031.stdout:1434 punch from 0x94b9ddf to 0x94bf90c, (0x5b2d bytes) 2024-09-13T07:12:48.846 INFO:teuthology.orchestra.run.smithi031.stdout:1437 clone 8 order 18 su 131072 sc 13 2024-09-13T07:12:48.915 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:48.916+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:48.973 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:48.973 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:48.973 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:48.973 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:48.973 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:48.982 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:49.008 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:49.008+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:49.109 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:49.109+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:914 3.14 3:2ed78eca:::rbd_data.1173f77ecad8.000000000000000a:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.888836723000001, 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082734499999999999, 2024-09-13T07:12:49.136 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 914 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246420+0000", 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246424+0000", 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246430+0000", 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8000000000000001e-06 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246434+0000", 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246477+0000", 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2443e-05 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246632+0000", 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015543100000000001 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254694+0000", 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080618979999999996 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1153 3.1b e25/19 3:d88532a6:::rbd_data.1173f77ecad8.000000000000015c:head v 25'14, mlcod=25'11)", 2024-09-13T07:12:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.807689858, 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091902019999999997, 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327567+0000", 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327572+0000", 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327579+0000", 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.306e-06 2024-09-13T07:12:49.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327585+0000", 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1279999999999996e-06 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327625+0000", 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9227999999999998e-05 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327641+0000", 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6059999999999999e-05 2024-09-13T07:12:49.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336708+0000", 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090672949999999995 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336758+0000", 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9870000000000002e-05 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1551 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'89, mlcod=25'86)", 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.357032555, 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077485289999999997, 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778224+0000", 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778233+0000", 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4689999999999997e-06 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778245+0000", 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2097e-05 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778252+0000", 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.934e-06 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778288+0000", 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6012e-05 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778302+0000", 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4630999999999999e-05 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785938+0000", 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076354329999999996 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785973+0000", 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5663000000000001e-05 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1552 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'90, mlcod=25'86)", 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.356384289999999, 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0071221499999999998, 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778873+0000", 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778887+0000", 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4409999999999999e-05 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778897+0000", 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0344e-05 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778903+0000", 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7019999999999998e-06 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778933+0000", 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0358e-05 2024-09-13T07:12:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778955+0000", 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1795999999999999e-05 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785976+0000", 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070212269999999997 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785995+0000", 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8907e-05 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.152 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1555 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'93, mlcod=25'86)", 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.354600467999999, 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0081821140000000007, 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780656+0000", 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780661+0000", 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.079e-06 2024-09-13T07:12:49.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780670+0000", 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6030000000000005e-06 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780676+0000", 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780720+0000", 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3767000000000003e-05 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780739+0000", 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9504000000000002e-05 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788759+0000", 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080197340000000006 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788839+0000", 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0075999999999997e-05 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1556 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'94, mlcod=25'86)", 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.35392371, 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0075812830000000003, 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781334+0000", 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781333+0000", 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781337+0000", 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781344+0000", 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781349+0000", 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781376+0000", 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7826000000000001e-05 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781392+0000", 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5401999999999999e-05 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788842+0000", 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007450684 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788915+0000", 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2556000000000001e-05 2024-09-13T07:12:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1557 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'95, mlcod=25'86)", 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.353332477, 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070583310000000002, 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781925+0000", 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781928+0000", 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4699999999999998e-06 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781934+0000", 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.302e-06 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781938+0000", 2024-09-13T07:12:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2559999999999998e-06 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781967+0000", 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9856e-05 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781979+0000", 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1857e-05 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788917+0000", 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069380750000000001 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788983+0000", 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5906000000000002e-05 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:2859 3.9 3:946400fb:::rbd_data.11734c6039c3.000000000000006d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5175725050000004, 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019805990999999998, 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 2859 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617683+0000", 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617695+0000", 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1626e-05 2024-09-13T07:12:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617723+0000", 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8252999999999999e-05 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617737+0000", 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3993e-05 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617824+0000", 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6908000000000002e-05 2024-09-13T07:12:49.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617958+0000", 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013417 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.621384+0000", 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034260499999999999 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637421+0000", 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016037264999999998 2024-09-13T07:12:49.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637430+0000", 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1549999999999996e-06 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637462+0000", 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1958000000000001e-05 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637491+0000", 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8523000000000001e-05 2024-09-13T07:12:49.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3313 3.1a 3:5f0039c6:::rbd_data.11734c6039c3.000000000000003e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.9375136770000001, 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021783180999999999, 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3313 2024-09-13T07:12:49.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197742+0000", 2024-09-13T07:12:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197754+0000", 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2519e-05 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197797+0000", 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3028999999999997e-05 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197803+0000", 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7420000000000003e-06 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197870+0000", 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7123000000000001e-05 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197959+0000", 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8409e-05 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.201633+0000", 2024-09-13T07:12:49.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036746299999999999 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219455+0000", 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017821452000000002 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219464+0000", 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8309999999999996e-06 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219498+0000", 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3948000000000001e-05 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219527+0000", 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8461000000000001e-05 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3991 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.176 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.676206+0000", 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.4590515660000003, 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070669260000000003, 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3991 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676206+0000", 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676206+0000", 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676205+0000", 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676208+0000", 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8439999999999998e-06 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676215+0000", 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4389999999999997e-06 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.676218+0000", 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.681655+0000", 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054377269999999998 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.681754+0000", 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8536000000000007e-05 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.682136+0000", 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038192800000000002 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683135+0000", 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00099909400000000007 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683222+0000", 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6916999999999995e-05 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683252+0000", 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0309000000000001e-05 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.683273+0000", 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0672000000000001e-05 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4059 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.3980134890000002, 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010501706, 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.182 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4059 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737245+0000", 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8199999999999999e-06 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737249+0000", 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5829999999999998e-06 2024-09-13T07:12:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737255+0000", 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4779999999999999e-06 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.738772+0000", 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001517853 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.738850+0000", 2024-09-13T07:12:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7206999999999994e-05 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739214+0000", 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036488900000000002 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739761+0000", 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054667599999999995 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747700+0000", 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0079386219999999993 2024-09-13T07:12:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747726+0000", 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5962999999999999e-05 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747746+0000", 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9899e-05 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4061 3.9 3:92d4a2e4:::rbd_data.11736f46b2f8.000000000000017d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.3953077020000002, 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078848679999999997, 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4061 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739949+0000", 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739954+0000", 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739968+0000", 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3419e-05 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739975+0000", 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3180000000000001e-06 2024-09-13T07:12:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740015+0000", 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9450999999999999e-05 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9463999999999999e-05 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.741328+0000", 2024-09-13T07:12:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00125416 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747778+0000", 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064493939999999998 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747786+0000", 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9060000000000002e-06 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747815+0000", 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9634000000000001e-05 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747835+0000", 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9395e-05 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:4059 3.9 e29/19 ondisk, result = 0)", 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.395183158, 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007677021, 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740077+0000", 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740085+0000", 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3459999999999994e-06 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740091+0000", 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5990000000000001e-06 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747692+0000", 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076006160000000001 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747697+0000", 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1560000000000003e-06 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747751+0000", 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4469999999999999e-05 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4348 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.1363173939999998, 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072710079999999998, 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.197 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4348 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998940+0000", 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998942+0000", 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7999999999999999e-06 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998945+0000", 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.998947+0000", 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.492e-06 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.001215+0000", 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022681929999999999 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.001296+0000", 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0691000000000004e-05 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.001639+0000", 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034265000000000001 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.002950+0000", 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001310877 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.006161+0000", 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00321193 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.006191+0000", 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9198999999999999e-05 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.006211+0000", 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0356999999999998e-05 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4383 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.1015203610000004, 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077768300000000002, 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4383 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033739+0000", 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6010000000000002e-06 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033743+0000", 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9709999999999998e-06 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033746+0000", 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035645+0000", 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001899097 2024-09-13T07:12:49.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035686+0000", 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0973000000000002e-05 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035859+0000", 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017274799999999999 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036847+0000", 2024-09-13T07:12:49.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00098787000000000007 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041442+0000", 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045950970000000002 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041477+0000", 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5629000000000001e-05 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041514+0000", 2024-09-13T07:12:49.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.65e-05 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4384 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.1014501689999996, 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077768220000000001, 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.208 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4384 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033805+0000", 2024-09-13T07:12:49.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033809+0000", 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1740000000000002e-06 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033814+0000", 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4479999999999996e-06 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033817+0000", 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.948e-06 2024-09-13T07:12:49.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036169+0000", 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023518459999999999 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036222+0000", 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3853000000000001e-05 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036439+0000", 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021612199999999999 2024-09-13T07:12:49.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037830+0000", 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013914540000000001 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041522+0000", 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036923220000000001 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041556+0000", 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4044000000000001e-05 2024-09-13T07:12:49.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041584+0000", 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7671e-05 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4385 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.1014127079999998, 2024-09-13T07:12:49.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078177509999999995, 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4385 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033844+0000", 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033847+0000", 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033855+0000", 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033859+0000", 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7110000000000001e-06 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036866+0000", 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030072530000000001 2024-09-13T07:12:49.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036917+0000", 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0992999999999998e-05 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037097+0000", 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018020300000000001 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.038207+0000", 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001109318 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041592+0000", 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003385047 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041631+0000", 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9196000000000002e-05 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041662+0000", 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1532000000000002e-05 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4386 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.1013476000000004, 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078157709999999991, 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.219 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4386 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033909+0000", 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033912+0000", 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4080000000000001e-06 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033918+0000", 2024-09-13T07:12:49.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5169999999999996e-06 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033922+0000", 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1599999999999998e-06 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037379+0000", 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034571020000000001 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037419+0000", 2024-09-13T07:12:49.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0751000000000003e-05 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037595+0000", 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017597700000000001 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.038494+0000", 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00089857399999999997 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041667+0000", 2024-09-13T07:12:49.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031729610000000002 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041699+0000", 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2005000000000003e-05 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041726+0000", 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6631e-05 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.9189854149999999, 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:7044 3.3 3:c3722553:::rbd_data.1173b1ea0ed4.0000000000000008:head [stat,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:48.690951+0000", 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.44430671799999999, 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0070132780000000004, 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 7044 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.690951+0000", 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.690951+0000", 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.690950+0000", 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.690952+0000", 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2190000000000001e-06 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.690955+0000", 2024-09-13T07:12:49.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.892e-06 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.690957+0000", 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4109999999999999e-06 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.690985+0000", 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8462999999999998e-05 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.691056+0000", 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0956000000000003e-05 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.691163+0000", 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106444 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.691704+0000", 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054100700000000003 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.697887+0000", 2024-09-13T07:12:49.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061832950000000001 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.697927+0000", 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9582000000000002e-05 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.697964+0000", 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7246000000000002e-05 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.235 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:49.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:12:49.237 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:49.237+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:49.238 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:49.293 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone7 from 0xe643f92 (overlap 0xd71e6f) to 0x0 2024-09-13T07:12:49.338 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:49.338+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:49.359 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T07:12:49.388 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:49.388 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:49.389 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:49.389 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:49.389 INFO:teuthology.orchestra.run.smithi031.stdout:1438 read 0x36512b7 thru 0x3660a43 (0xf78d bytes) 2024-09-13T07:12:49.389 INFO:teuthology.orchestra.run.smithi031.stdout:1439 write 0xcb883fb thru 0xcb958f4 (0xd4fa bytes) 2024-09-13T07:12:49.396 INFO:teuthology.orchestra.run.smithi031.stdout:1440 read 0x8556c68 thru 0x85622a2 (0xb63b bytes) 2024-09-13T07:12:49.396 INFO:teuthology.orchestra.run.smithi031.stdout:1441 read 0xd1725b1 thru 0xd1804c7 (0xdf17 bytes) 2024-09-13T07:12:49.396 INFO:teuthology.orchestra.run.smithi031.stdout:1443 punch from 0xafe8d0b to 0xafe9bed, (0xee2 bytes) 2024-09-13T07:12:49.396 INFO:teuthology.orchestra.run.smithi031.stdout:1444 write 0x1612164 thru 0x161d49d (0xb33a bytes) 2024-09-13T07:12:49.397 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:49.399 INFO:teuthology.orchestra.run.smithi031.stdout:1445 read 0xe425067 thru 0xe42766c (0x2606 bytes) 2024-09-13T07:12:49.399 INFO:teuthology.orchestra.run.smithi031.stdout:1447 read 0xb73e7ca thru 0xb7432c1 (0x4af8 bytes) 2024-09-13T07:12:49.404 INFO:teuthology.orchestra.run.smithi031.stdout:1449 write 0xe926b62 thru 0xe92ec1f (0x80be bytes) 2024-09-13T07:12:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:1450 read 0x1a14615 thru 0x1a1469d (0x89 bytes) 2024-09-13T07:12:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:1451 write 0x40cc246 thru 0x40d8b35 (0xc8f0 bytes) 2024-09-13T07:12:49.421 INFO:teuthology.orchestra.run.smithi031.stdout:1452 trunc from 0xe92ec20 to 0x4f61ca6 2024-09-13T07:12:49.439 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:49.440+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:49.444 INFO:teuthology.orchestra.run.smithi031.stdout:1453 read 0x9455b7 thru 0x94e530 (0x8f7a bytes) 2024-09-13T07:12:49.444 INFO:teuthology.orchestra.run.smithi031.stdout:1454 read 0x27a2c34 thru 0x27af5f6 (0xc9c3 bytes) 2024-09-13T07:12:49.445 INFO:teuthology.orchestra.run.smithi031.stdout:1455 read 0x4838e29 thru 0x483d96a (0x4b42 bytes) 2024-09-13T07:12:49.445 INFO:teuthology.orchestra.run.smithi031.stdout:1456 trunc from 0x4f61ca6 to 0x5408160 2024-09-13T07:12:49.461 INFO:teuthology.orchestra.run.smithi031.stdout:1458 write 0xb4b2ecd thru 0xb4bee26 (0xbf5a bytes) 2024-09-13T07:12:49.465 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.465 INFO:teuthology.orchestra.run.smithi031.stdout:1460 read 0x6fd0fc2 thru 0x6fd2774 (0x17b3 bytes) 2024-09-13T07:12:49.465 INFO:teuthology.orchestra.run.smithi031.stdout:1461 writesame 0x4762a52 thru 0x4763aca data_size 0x1(0x1079 bytes) 2024-09-13T07:12:49.468 INFO:teuthology.orchestra.run.smithi031.stdout:1462 read 0x5dc7a9a thru 0x5dd0cf0 (0x9257 bytes) 2024-09-13T07:12:49.471 INFO:teuthology.orchestra.run.smithi031.stdout:1463 trunc from 0xb4bee27 to 0x8aaf3cd 2024-09-13T07:12:49.481 INFO:teuthology.orchestra.run.smithi031.stdout:1464 read 0x65c3caf thru 0x65c727b (0x35cd bytes) 2024-09-13T07:12:49.488 INFO:teuthology.orchestra.run.smithi031.stdout:1465 trunc from 0x8aaf3cd to 0x330306e 2024-09-13T07:12:49.505 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:49.505 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:49.505 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:49.505 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:49.505 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:1467 punch from 0x25bf82c to 0x25c9b2b, (0xa2ff bytes) 2024-09-13T07:12:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:1468 read 0xadd2d5 thru 0xae8455 (0xb181 bytes) 2024-09-13T07:12:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:1469 writesame 0x109af45 thru 0x109e215 data_size 0x1(0x32d1 bytes) 2024-09-13T07:12:49.514 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:49.518 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.528 INFO:teuthology.orchestra.run.smithi031.stdout:1471 write 0x6ff5372 thru 0x6ffd58e (0x821d bytes) 2024-09-13T07:12:49.531 INFO:teuthology.orchestra.run.smithi031.stdout:1472 punch from 0x55e398c to 0x55f0e66, (0xd4da bytes) 2024-09-13T07:12:49.531 INFO:teuthology.orchestra.run.smithi031.stdout:1473 read 0x5f77273 thru 0x5f7a0f3 (0x2e81 bytes) 2024-09-13T07:12:49.537 INFO:teuthology.orchestra.run.smithi031.stdout:1475 write 0x7cd5eb7 thru 0x7cdc987 (0x6ad1 bytes) 2024-09-13T07:12:49.540 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.545 INFO:teuthology.orchestra.run.smithi031.stdout:1481 trunc from 0x7cdc988 to 0xc678052 2024-09-13T07:12:49.550 INFO:teuthology.orchestra.run.smithi031.stdout:1482 punch from 0x47c8d42 to 0x47cd09c, (0x435a bytes) 2024-09-13T07:12:49.551 INFO:teuthology.orchestra.run.smithi031.stdout:1483 write 0x2c44407 thru 0x2c45eb5 (0x1aaf bytes) 2024-09-13T07:12:49.554 INFO:teuthology.orchestra.run.smithi031.stdout:1485 punch from 0x8ac9fbb to 0x8aceafc, (0x4b41 bytes) 2024-09-13T07:12:49.555 INFO:teuthology.orchestra.run.smithi031.stdout:1486 trunc from 0xc678052 to 0xd1ebcda 2024-09-13T07:12:49.559 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:49.560+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:49.560 INFO:teuthology.orchestra.run.smithi031.stdout:1487 write 0x88869a3 thru 0x8893df2 (0xd450 bytes) 2024-09-13T07:12:49.563 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.563 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.563 INFO:teuthology.orchestra.run.smithi031.stdout:1494 write 0xd1293b3 thru 0xd1392eb (0xff39 bytes) 2024-09-13T07:12:49.567 INFO:teuthology.orchestra.run.smithi031.stdout:1495 writesame 0x2fe1583 thru 0x2fe2104 data_size 0x3(0xb82 bytes) 2024-09-13T07:12:49.570 INFO:teuthology.orchestra.run.smithi031.stdout:1496 punch from 0x640ff5b to 0x64191ee, (0x9293 bytes) 2024-09-13T07:12:49.570 INFO:teuthology.orchestra.run.smithi031.stdout:1497 read 0xc8933d6 thru 0xc89d36b (0x9f96 bytes) 2024-09-13T07:12:49.570 INFO:teuthology.orchestra.run.smithi031.stdout:1498 read 0x537a716 thru 0x53890b9 (0xe9a4 bytes) 2024-09-13T07:12:49.570 INFO:teuthology.orchestra.run.smithi031.stdout:1499 writesame 0x68c9a5c thru 0x68d84e6 data_size 0x1(0xea8b bytes) 2024-09-13T07:12:49.601 INFO:teuthology.orchestra.run.smithi031.stdout:1500 read 0x87d4a7c thru 0x87e1705 (0xcc8a bytes) 2024-09-13T07:12:49.601 INFO:teuthology.orchestra.run.smithi031.stdout:1501 punch from 0x277b20e to 0x278550a, (0xa2fc bytes) 2024-09-13T07:12:49.601 INFO:teuthology.orchestra.run.smithi031.stdout:1502 writesame 0xa46382d thru 0xa4707d3 data_size 0x35(0xcfa7 bytes) 2024-09-13T07:12:49.604 INFO:teuthology.orchestra.run.smithi031.stdout:1504 read 0x8673ebc thru 0x86742c0 (0x405 bytes) 2024-09-13T07:12:49.604 INFO:teuthology.orchestra.run.smithi031.stdout:1505 write 0x8944e9 thru 0x89ee86 (0xa99e bytes) 2024-09-13T07:12:49.610 INFO:teuthology.orchestra.run.smithi031.stdout:1506 writesame 0xcd4dd1f thru 0xcd5aac9 data_size 0x1(0xcdab bytes) 2024-09-13T07:12:49.623 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:49.623 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:49.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:49.623 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1298 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161601968999999, 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01046212, 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1298 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461386+0000", 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461385+0000", 2024-09-13T07:12:49.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461390+0000", 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5210000000000004e-06 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461394+0000", 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2230000000000001e-06 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461397+0000", 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6910000000000002e-06 2024-09-13T07:12:49.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464234+0000", 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028368820000000002 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464318+0000", 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4067000000000007e-05 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.464664+0000", 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034640300000000001 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468940+0000", 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042761190000000001 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471764+0000", 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002823739 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471807+0000", 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3485999999999997e-05 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471848+0000", 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0392999999999998e-05 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1300 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.16156878, 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010517047999999999, 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.629 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1300 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461419+0000", 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461418+0000", 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461421+0000", 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3489999999999999e-06 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461425+0000", 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8330000000000004e-06 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461427+0000", 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5519999999999999e-06 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465053+0000", 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036258309999999999 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465104+0000", 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0825999999999998e-05 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465312+0000", 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020849900000000001 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468969+0000", 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036562119999999998 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471860+0000", 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002891841 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471903+0000", 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2315999999999999e-05 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471936+0000", 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3132000000000001e-05 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.634 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1301 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161538439999999, 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010564656, 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.635 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1301 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461449+0000", 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461452+0000", 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4690000000000002e-06 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461456+0000", 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0810000000000004e-06 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461459+0000", 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5399999999999998e-06 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465574+0000", 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041147930000000003 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.465655+0000", 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1254999999999994e-05 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466023+0000", 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036823900000000002 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468992+0000", 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029688190000000001 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471945+0000", 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029525609999999998 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471982+0000", 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7837999999999999e-05 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472014+0000", 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1432e-05 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.640 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1302 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161508221, 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010611844, 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1302 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461479+0000", 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461481+0000", 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4040000000000002e-06 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461485+0000", 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7969999999999998e-06 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461488+0000", 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6400000000000001e-06 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466476+0000", 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049885140000000003 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466555+0000", 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8208999999999996e-05 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.466936+0000", 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038099899999999998 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469019+0000", 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020838300000000001 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472021+0000", 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030016019999999999 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472056+0000", 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5295e-05 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472091+0000", 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4876999999999998e-05 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1303 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161475926, 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010655628, 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.646 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1303 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461512+0000", 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461511+0000", 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461514+0000", 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.418e-06 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461518+0000", 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9219999999999999e-06 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461520+0000", 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.57e-06 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467284+0000", 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057634100000000001 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467336+0000", 2024-09-13T07:12:49.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2633999999999998e-05 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.467664+0000", 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00032754700000000003 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471710+0000", 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004046231 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472100+0000", 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038953300000000001 2024-09-13T07:12:49.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472136+0000", 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6699000000000002e-05 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472167+0000", 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1019999999999998e-05 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1304 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.651 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161450821000001, 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010707311000000001, 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1304 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:49.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461537+0000", 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461536+0000", 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461539+0000", 2024-09-13T07:12:49.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.244e-06 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461543+0000", 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7099999999999998e-06 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461546+0000", 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5170000000000002e-06 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468072+0000", 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065260650000000002 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468137+0000", 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5036999999999994e-05 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.468389+0000", 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025219400000000002 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471754+0000", 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033649280000000001 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000421183 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472212+0000", 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7153999999999997e-05 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472244+0000", 2024-09-13T07:12:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1612999999999997e-05 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1305 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161419662, 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010819396, 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.657 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1305 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461568+0000", 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461570+0000", 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3860000000000001e-06 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461574+0000", 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461577+0000", 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6340000000000001e-06 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469041+0000", 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0074637030000000004 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469091+0000", 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0197000000000001e-05 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469298+0000", 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020718900000000001 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472253+0000", 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002954702 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472306+0000", 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3198999999999997e-05 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472351+0000", 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000001e-05 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472387+0000", 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6145e-05 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1306 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.662 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161388992999999, 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010918835999999999, 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1306 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461599+0000", 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461598+0000", 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461602+0000", 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0860000000000004e-06 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461610+0000", 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0830000000000004e-06 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461616+0000", 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7379999999999999e-06 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469601+0000", 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0079851690000000003 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469669+0000", 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8316000000000001e-05 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.469866+0000", 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000196768 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472259+0000", 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002392786 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472441+0000", 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018190200000000001 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472485+0000", 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3853999999999997e-05 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472517+0000", 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2614999999999999e-05 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1307 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.161343826, 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0110288, 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.668 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1307 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.669 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461644+0000", 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461643+0000", 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461647+0000", 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9469999999999996e-06 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461656+0000", 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9279999999999999e-06 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.461664+0000", 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5989999999999996e-06 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470152+0000", 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0084883609999999998 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470233+0000", 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1287999999999998e-05 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470632+0000", 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00039872099999999999 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472531+0000", 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018986420000000001 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472593+0000", 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2902000000000002e-05 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472637+0000", 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3300000000000002e-05 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472673+0000", 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5839e-05 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1308 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.160900218, 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011378667, 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1308 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462087+0000", 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462086+0000", 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462092+0000", 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9020000000000004e-06 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462104+0000", 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1807999999999999e-05 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.462113+0000", 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7560000000000005e-06 2024-09-13T07:12:49.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470891+0000", 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008778569 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.470955+0000", 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3859e-05 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.471342+0000", 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038666200000000002 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473305+0000", 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019625630000000001 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473376+0000", 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1354999999999997e-05 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473424+0000", 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8010000000000003e-05 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473466+0000", 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2132000000000003e-05 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1309 3.1d 3:bcefd1cd:::rbd_data.1173c53667c.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:49.679 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.159241779, 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014288309000000001, 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1309 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463745+0000", 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463751+0000", 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:12:49.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463761+0000", 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3589999999999997e-06 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463767+0000", 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4420000000000003e-06 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472731+0000", 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0089641479999999999 2024-09-13T07:12:49.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472858+0000", 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012669100000000001 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473057+0000", 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019875900000000001 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477793+0000", 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047361900000000004 2024-09-13T07:12:49.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477983+0000", 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019018299999999999 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478014+0000", 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1062999999999997e-05 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478034+0000", 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.978e-05 2024-09-13T07:12:49.684 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1373 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.099761322999999, 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010609936, 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.685 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1373 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.686 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523226+0000", 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523231+0000", 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9280000000000001e-06 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523238+0000", 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3509999999999996e-06 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.687 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523243+0000", 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0900000000000004e-06 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.527993+0000", 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047498699999999998 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.528077+0000", 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3616999999999996e-05 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.688 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.528470+0000", 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000393853 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532078+0000", 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036080019999999999 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533743+0000", 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001664998 2024-09-13T07:12:49.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533790+0000", 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6326000000000003e-05 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533836+0000", 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6490999999999999e-05 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.690 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1378 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.099101455, 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010563678, 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1378 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.691 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523886+0000", 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.692 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523889+0000", 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2499999999999998e-06 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523896+0000", 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.523901+0000", 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8679999999999998e-06 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.693 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532169+0000", 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082678690000000006 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532251+0000", 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2207999999999996e-05 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532620+0000", 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036913700000000001 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534210+0000", 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015899690000000001 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534377+0000", 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00016627800000000001 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534416+0000", 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9626000000000003e-05 2024-09-13T07:12:49.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534450+0000", 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3482999999999998e-05 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1379 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.098975161, 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011548345999999999, 2024-09-13T07:12:49.696 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1379 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524012+0000", 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524016+0000", 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.703e-06 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524026+0000", 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9909999999999997e-06 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524031+0000", 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9190000000000004e-06 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.532975+0000", 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.008944044 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533061+0000", 2024-09-13T07:12:49.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5353999999999996e-05 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.533391+0000", 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00032971500000000001 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534330+0000", 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00093939800000000003 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535481+0000", 2024-09-13T07:12:49.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011505700000000001 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535522+0000", 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1958e-05 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.535561+0000", 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8278999999999997e-05 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1380 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.098770786999999, 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012748349000000001, 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1380 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524217+0000", 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524216+0000", 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524221+0000", 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4519999999999996e-06 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524233+0000", 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1652e-05 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.524241+0000", 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1100000000000003e-06 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534487+0000", 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010246207 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534570+0000", 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2948999999999995e-05 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.534908+0000", 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00033817199999999997 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536427+0000", 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015188420000000001 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536868+0000", 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00044121999999999998 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536920+0000", 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1520999999999997e-05 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.536965+0000", 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5160999999999997e-05 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1555 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.845493392, 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011519636, 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.707 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1555 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777494+0000", 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777497+0000", 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5609999999999999e-06 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777504+0000", 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5220000000000004e-06 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777509+0000", 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7890000000000002e-06 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780260+0000", 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0027512460000000002 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780343+0000", 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3100000000000001e-05 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780598+0000", 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025544100000000002 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786194+0000", 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055953080000000002 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788934+0000", 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002739806 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788978+0000", 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4274000000000003e-05 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789014+0000", 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6010999999999998e-05 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.712 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1556 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.845473469, 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011620324, 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1556 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.713 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777514+0000", 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777517+0000", 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7980000000000001e-06 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777524+0000", 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2899999999999997e-06 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777529+0000", 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8319999999999996e-06 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781008+0000", 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034795770000000002 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781090+0000", 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1499e-05 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781297+0000", 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000207689 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786431+0000", 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051334140000000002 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789071+0000", 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026401670000000001 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789103+0000", 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2400999999999997e-05 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789134+0000", 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0967999999999999e-05 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1557 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.84543311, 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011717251999999999, 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1557 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.719 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777554+0000", 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777558+0000", 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2869999999999998e-06 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777568+0000", 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7680000000000003e-06 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777574+0000", 2024-09-13T07:12:49.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6520000000000003e-06 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781632+0000", 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040581599999999999 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781679+0000", 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6934000000000002e-05 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.781867+0000", 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018818900000000001 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786435+0000", 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045683039999999996 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789187+0000", 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002752151 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789232+0000", 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.494e-05 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789272+0000", 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9376999999999999e-05 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1558 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.845392985, 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011767046999999999, 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1558 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777595+0000", 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777594+0000", 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777597+0000", 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2140000000000001e-06 2024-09-13T07:12:49.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777605+0000", 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5900000000000002e-06 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777610+0000", 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782276+0000", 2024-09-13T07:12:49.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0046664000000000002 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782321+0000", 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4489e-05 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782507+0000", 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018627699999999999 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.786438+0000", 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039313780000000001 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789315+0000", 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002876871 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789342+0000", 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6415e-05 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789362+0000", 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9868999999999999e-05 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1559 3.1d 3:b82b9b5d:::rbd_object_map.1173c53667c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.845303712, 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011750661000000001, 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:49.730 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1559 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777684+0000", 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777683+0000", 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777690+0000", 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8730000000000001e-06 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777703+0000", 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3016999999999999e-05 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.777710+0000", 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6240000000000002e-06 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782759+0000", 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050484909999999996 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.782845+0000", 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6082999999999996e-05 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.783061+0000", 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000215732 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788822+0000", 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057617689999999999 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789389+0000", 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00056687899999999995 2024-09-13T07:12:49.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789416+0000", 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7132999999999999e-05 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.789435+0000", 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8199999999999999e-05 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:49.735 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:49.738 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:49.739 INFO:teuthology.orchestra.run.smithi031.stdout:1509 read 0x968a81a thru 0x968cac9 (0x22b0 bytes) 2024-09-13T07:12:49.739 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.739 INFO:teuthology.orchestra.run.smithi031.stdout:1512 read 0xa673682 thru 0xa67c2a0 (0x8c1f bytes) 2024-09-13T07:12:49.739 INFO:teuthology.orchestra.run.smithi031.stdout:1515 write 0x75164b4 thru 0x7522adc (0xc629 bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1516 write 0x668629b thru 0x6687a84 (0x17ea bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1518 read 0x4e5da89 thru 0x4e68782 (0xacfa bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1520 trunc from 0xd1ebcda to 0x223efbd 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1521 write 0xd204e44 thru 0xd2122ff (0xd4bc bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1523 trunc from 0xd212300 to 0xe8e8707 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1525 read 0x89e110c thru 0x89ef20b (0xe100 bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1526 writesame 0xd0221ab thru 0xd02bc4e data_size 0x6(0x9aa4 bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1527 punch from 0x1734bb5 to 0x173ab09, (0x5f54 bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1528 write 0x90f12e0 thru 0x90f3021 (0x1d42 bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1529 write 0xc6978db thru 0xc6a6dea (0xf510 bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1531 read 0x155a454 thru 0x1561620 (0x71cd bytes) 2024-09-13T07:12:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:1532 writesame 0x36e3cdd thru 0x36f0fef data_size 0x65(0xd313 bytes) 2024-09-13T07:12:49.741 INFO:teuthology.orchestra.run.smithi031.stdout:1533 writesame 0xb44885e thru 0xb456373 data_size 0x1d(0xdb16 bytes) 2024-09-13T07:12:49.741 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.741 INFO:teuthology.orchestra.run.smithi031.stdout:1535 punch from 0xacdb373 to 0xace16b6, (0x6343 bytes) 2024-09-13T07:12:49.741 INFO:teuthology.orchestra.run.smithi031.stdout:1536 read 0x7242d80 thru 0x7246309 (0x358a bytes) 2024-09-13T07:12:49.741 INFO:teuthology.orchestra.run.smithi031.stdout:1538 writesame 0x7853bed thru 0x785ce32 data_size 0x4f(0x9246 bytes) 2024-09-13T07:12:49.741 INFO:teuthology.orchestra.run.smithi031.stdout:1540 punch from 0xca3036e to 0xca320ce, (0x1d60 bytes) 2024-09-13T07:12:49.741 INFO:teuthology.orchestra.run.smithi031.stdout:1541 write 0xc197bbf thru 0xc19f264 (0x76a6 bytes) 2024-09-13T07:12:49.741 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:49.738+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:49.742 INFO:teuthology.orchestra.run.smithi031.stdout:1542 trunc from 0xe8e8707 to 0x6378d5a 2024-09-13T07:12:49.750 INFO:teuthology.orchestra.run.smithi031.stdout:1543 punch from 0x3d831d0 to 0x3d847ae, (0x15de bytes) 2024-09-13T07:12:49.750 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.750 INFO:teuthology.orchestra.run.smithi031.stdout:1545 punch from 0x5eb0601 to 0x5ebfd6a, (0xf769 bytes) 2024-09-13T07:12:49.750 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.750 INFO:teuthology.orchestra.run.smithi031.stdout:1550 read 0x26c877a thru 0x26cca6f (0x42f6 bytes) 2024-09-13T07:12:49.760 INFO:teuthology.orchestra.run.smithi031.stdout:1552 write 0xb0daf63 thru 0xb0e5ab4 (0xab52 bytes) 2024-09-13T07:12:49.766 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.766 INFO:teuthology.orchestra.run.smithi031.stdout:1554 write 0x9e6f444 thru 0x9e7099f (0x155c bytes) 2024-09-13T07:12:49.768 INFO:teuthology.orchestra.run.smithi031.stdout:1556 write 0x7a742f8 thru 0x7a7fee6 (0xbbef bytes) 2024-09-13T07:12:49.770 INFO:teuthology.orchestra.run.smithi031.stdout:1557 write 0x2c44d6a thru 0x2c4a16f (0x5406 bytes) 2024-09-13T07:12:49.773 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.773 INFO:teuthology.orchestra.run.smithi031.stdout:1559 write 0x92a6507 thru 0x92b086d (0xa367 bytes) 2024-09-13T07:12:49.776 INFO:teuthology.orchestra.run.smithi031.stdout:1560 punch from 0x65cf62e to 0x65da08e, (0xaa60 bytes) 2024-09-13T07:12:49.776 INFO:teuthology.orchestra.run.smithi031.stdout:1561 write 0x7c37dfa thru 0x7c3c7ee (0x49f5 bytes) 2024-09-13T07:12:49.785 INFO:teuthology.orchestra.run.smithi031.stdout:1562 trunc from 0xb0e5ab5 to 0x5ec18bf 2024-09-13T07:12:49.796 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.796 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.796 INFO:teuthology.orchestra.run.smithi031.stdout:1566 punch from 0x4c2f9bb to 0x4c39819, (0x9e5e bytes) 2024-09-13T07:12:49.805 INFO:teuthology.orchestra.run.smithi031.stdout:1569 writesame 0x919dc8b thru 0x91acb2e data_size 0x12(0xeea4 bytes) 2024-09-13T07:12:49.805 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.806 INFO:teuthology.orchestra.run.smithi031.stdout:1571 write 0x6c465a0 thru 0x6c4be97 (0x58f8 bytes) 2024-09-13T07:12:49.807 INFO:teuthology.orchestra.run.smithi184.stdout: marking collection for removal 2024-09-13T07:12:49.807 INFO:teuthology.orchestra.run.smithi184.stdout:setting '_remove' omap key 2024-09-13T07:12:49.807 INFO:teuthology.orchestra.run.smithi184.stdout:finish_remove_pgs 3.16_head removing 3.16 2024-09-13T07:12:49.808 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:49.808 INFO:teuthology.orchestra.run.smithi031.stdout:1573 write 0x8cc3fac thru 0x8cd2c45 (0xec9a bytes) 2024-09-13T07:12:49.808 INFO:teuthology.orchestra.run.smithi184.stdout:Remove successful 2024-09-13T07:12:49.810 INFO:teuthology.orchestra.run.smithi031.stdout:1575 writesame 0x1a319b7 thru 0x1a36739 data_size 0x1(0x4d83 bytes) 2024-09-13T07:12:49.822 INFO:teuthology.orchestra.run.smithi031.stdout:1576 read 0x85d7d2d thru 0x85df5a9 (0x787d bytes) 2024-09-13T07:12:49.830 INFO:teuthology.orchestra.run.smithi031.stdout:1578 trunc from 0x91acb2f to 0x1a4a14a 2024-09-13T07:12:49.837 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:49.838+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:49.847 INFO:teuthology.orchestra.run.smithi031.stdout:1579 read 0x19e1a86 thru 0x19f027e (0xe7f9 bytes) 2024-09-13T07:12:49.848 INFO:teuthology.orchestra.run.smithi031.stdout:1580 punch from 0x11deb7 to 0x127a40, (0x9b89 bytes) 2024-09-13T07:12:49.855 INFO:teuthology.orchestra.run.smithi031.stdout:1581 trunc from 0x1a4a14a to 0x7c1c601 2024-09-13T07:12:49.873 INFO:teuthology.orchestra.run.smithi031.stdout:1582 write 0xe4aa7a4 thru 0xe4b8a08 (0xe265 bytes) 2024-09-13T07:12:49.876 INFO:teuthology.orchestra.run.smithi031.stdout:1583 write 0xe418870 thru 0xe427d4a (0xf4db bytes) 2024-09-13T07:12:49.881 INFO:teuthology.orchestra.run.smithi031.stdout:1584 punch from 0xbde760d to 0xbdf271a, (0xb10d bytes) 2024-09-13T07:12:49.881 INFO:teuthology.orchestra.run.smithi031.stdout:1585 writesame 0xa78456d thru 0xa789aaa data_size 0x3(0x553e bytes) 2024-09-13T07:12:49.888 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:49.888 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:49.888 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:49.888 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:49.897 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:49.901 INFO:teuthology.orchestra.run.smithi031.stdout:1586 trunc from 0xe4b8a09 to 0xc1c616 2024-09-13T07:12:49.928 INFO:teuthology.orchestra.run.smithi031.stdout:1589 trunc from 0xc1c616 to 0x7c8f259 2024-09-13T07:12:49.936 INFO:teuthology.orchestra.run.smithi031.stdout:1591 trunc from 0x7c8f259 to 0xb11bf9f 2024-09-13T07:12:49.938 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:49.939+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:49.948 INFO:teuthology.orchestra.run.smithi031.stdout:1592 write 0xc46e408 thru 0xc471425 (0x301e bytes) 2024-09-13T07:12:49.951 INFO:teuthology.orchestra.run.smithi031.stdout:1593 punch from 0x65559bb to 0x6560631, (0xac76 bytes) 2024-09-13T07:12:49.951 INFO:teuthology.orchestra.run.smithi031.stdout:1594 punch from 0x1308fe1 to 0x130d786, (0x47a5 bytes) 2024-09-13T07:12:49.953 INFO:teuthology.orchestra.run.smithi031.stdout:1595 trunc from 0xc471426 to 0x9e163fa 2024-09-13T07:12:49.962 INFO:teuthology.orchestra.run.smithi031.stdout:1596 punch from 0x17a448d to 0x17b20d0, (0xdc43 bytes) 2024-09-13T07:12:49.962 INFO:teuthology.orchestra.run.smithi031.stdout:1598 writesame 0x290947a thru 0x2918616 data_size 0x1(0xf19d bytes) 2024-09-13T07:12:49.985 INFO:teuthology.orchestra.run.smithi031.stdout:1599 read 0x2d79b44 thru 0x2d7e6e8 (0x4ba5 bytes) 2024-09-13T07:12:49.985 INFO:teuthology.orchestra.run.smithi031.stdout:1601 writesame 0x7326a95 thru 0x73316a6 data_size 0x19(0xac12 bytes) 2024-09-13T07:12:49.989 INFO:teuthology.orchestra.run.smithi031.stdout:1603 write 0x319193c thru 0x31a0168 (0xe82d bytes) 2024-09-13T07:12:49.992 INFO:teuthology.orchestra.run.smithi031.stdout:1605 read 0x824b42b thru 0x824ecf2 (0x38c8 bytes) 2024-09-13T07:12:49.992 INFO:teuthology.orchestra.run.smithi031.stdout:1606 write 0x7f287eb thru 0x7f33281 (0xaa97 bytes) 2024-09-13T07:12:49.995 INFO:teuthology.orchestra.run.smithi031.stdout:1607 write 0x44e0065 thru 0x44e5db4 (0x5d50 bytes) 2024-09-13T07:12:50.005 INFO:teuthology.orchestra.run.smithi031.stdout:1608 trunc from 0x9e163fa to 0x338121e 2024-09-13T07:12:50.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.025 INFO:teuthology.orchestra.run.smithi031.stdout:1610 write 0x864b1dc thru 0x864d968 (0x278d bytes) 2024-09-13T07:12:50.027 INFO:teuthology.orchestra.run.smithi031.stdout:1611 read 0x6c307e6 thru 0x6c3c5dc (0xbdf7 bytes) 2024-09-13T07:12:50.027 INFO:teuthology.orchestra.run.smithi031.stdout:1612 write 0x2db7938 thru 0x2dc75de (0xfca7 bytes) 2024-09-13T07:12:50.031 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.036 INFO:teuthology.orchestra.run.smithi031.stdout:1614 trunc from 0x864d969 to 0x3812336 2024-09-13T07:12:50.038 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:50.039+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:50.042 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:50.043 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:50.043 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:50.043 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:50.043 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:50.045 INFO:teuthology.orchestra.run.smithi031.stdout:1615 punch from 0xbf62bd to 0xc01da5, (0xbae8 bytes) 2024-09-13T07:12:50.045 INFO:teuthology.orchestra.run.smithi031.stdout:1616 punch from 0x3023da3 to 0x3024729, (0x986 bytes) 2024-09-13T07:12:50.045 INFO:teuthology.orchestra.run.smithi031.stdout:1617 punch from 0x1e7fc07 to 0x1e8fbdb, (0xffd4 bytes) 2024-09-13T07:12:50.049 INFO:teuthology.orchestra.run.smithi031.stdout:1618 trunc from 0x3812336 to 0x75519e2 2024-09-13T07:12:50.052 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:50.054 INFO:teuthology.orchestra.run.smithi031.stdout:1619 punch from 0x2298f66 to 0x229c31a, (0x33b4 bytes) 2024-09-13T07:12:50.061 INFO:teuthology.orchestra.run.smithi031.stdout:1620 trunc from 0x75519e2 to 0xd86c328 2024-09-13T07:12:50.067 INFO:teuthology.orchestra.run.smithi031.stdout:1621 read 0x697017d thru 0x6971c6f (0x1af3 bytes) 2024-09-13T07:12:50.067 INFO:teuthology.orchestra.run.smithi031.stdout:1622 read 0x8a8dfa9 thru 0x8a9a630 (0xc688 bytes) 2024-09-13T07:12:50.067 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.067 INFO:teuthology.orchestra.run.smithi031.stdout:1624 read 0x7f274a0 thru 0x7f359e6 (0xe547 bytes) 2024-09-13T07:12:50.067 INFO:teuthology.orchestra.run.smithi031.stdout:1626 writesame 0x58f8dd6 thru 0x590026c data_size 0x1(0x7497 bytes) 2024-09-13T07:12:50.083 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.083 INFO:teuthology.orchestra.run.smithi031.stdout:1628 punch from 0x7091596 to 0x7097a93, (0x64fd bytes) 2024-09-13T07:12:50.083 INFO:teuthology.orchestra.run.smithi031.stdout:1629 read 0x54eee32 thru 0x54f68eb (0x7aba bytes) 2024-09-13T07:12:50.083 INFO:teuthology.orchestra.run.smithi031.stdout:1631 writesame 0x11ef485 thru 0x11ef8a3 data_size 0x1(0x41f bytes) 2024-09-13T07:12:50.086 INFO:teuthology.orchestra.run.smithi031.stdout:1635 punch from 0x7bda392 to 0x7bdccc3, (0x2931 bytes) 2024-09-13T07:12:50.086 INFO:teuthology.orchestra.run.smithi031.stdout:1637 read 0x3b2e45d thru 0x3b3b6de (0xd282 bytes) 2024-09-13T07:12:50.086 INFO:teuthology.orchestra.run.smithi031.stdout:1638 write 0x737f7a5 thru 0x738931f (0x9b7b bytes) 2024-09-13T07:12:50.088 INFO:teuthology.orchestra.run.smithi031.stdout:1639 writesame 0x6a1dce0 thru 0x6a28388 data_size 0x73(0xa6a9 bytes) 2024-09-13T07:12:50.092 INFO:teuthology.orchestra.run.smithi031.stdout:1640 write 0xaca2443 thru 0xacad328 (0xaee6 bytes) 2024-09-13T07:12:50.094 INFO:teuthology.orchestra.run.smithi031.stdout:1643 punch from 0x807649e to 0x80818e6, (0xb448 bytes) 2024-09-13T07:12:50.095 INFO:teuthology.orchestra.run.smithi031.stdout:1646 write 0x5862ba2 thru 0x5871e28 (0xf287 bytes) 2024-09-13T07:12:50.097 INFO:teuthology.orchestra.run.smithi031.stdout:1647 write 0x480d487 thru 0x481c69b (0xf215 bytes) 2024-09-13T07:12:50.100 INFO:teuthology.orchestra.run.smithi031.stdout:1651 write 0x5ade778 thru 0x5ae2b9d (0x4426 bytes) 2024-09-13T07:12:50.104 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.104 INFO:teuthology.orchestra.run.smithi031.stdout:1653 punch from 0x60c2eb8 to 0x60c4c45, (0x1d8d bytes) 2024-09-13T07:12:50.104 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.105 INFO:teuthology.orchestra.run.smithi031.stdout:1660 write 0x545eb6c thru 0x5469890 (0xad25 bytes) 2024-09-13T07:12:50.107 INFO:teuthology.orchestra.run.smithi031.stdout:1662 read 0x75e5ca8 thru 0x75e708b (0x13e4 bytes) 2024-09-13T07:12:50.107 INFO:teuthology.orchestra.run.smithi031.stdout:1663 punch from 0xd562dda to 0xd568e91, (0x60b7 bytes) 2024-09-13T07:12:50.107 INFO:teuthology.orchestra.run.smithi031.stdout:1665 write 0x31922a4 thru 0x319f5b6 (0xd313 bytes) 2024-09-13T07:12:50.109 INFO:teuthology.orchestra.run.smithi031.stdout:1667 write 0x21b2ac0 thru 0x21b790f (0x4e50 bytes) 2024-09-13T07:12:50.112 INFO:teuthology.orchestra.run.smithi031.stdout:1668 trunc from 0xd86c328 to 0xcdb7164 2024-09-13T07:12:50.118 INFO:teuthology.orchestra.run.smithi031.stdout:1669 read 0x4fca8f6 thru 0x4fd15b7 (0x6cc2 bytes) 2024-09-13T07:12:50.118 INFO:teuthology.orchestra.run.smithi031.stdout:1670 write 0xc4d9d3c thru 0xc4da45e (0x723 bytes) 2024-09-13T07:12:50.126 INFO:teuthology.orchestra.run.smithi031.stdout:1671 write 0xd651e52 thru 0xd659220 (0x73cf bytes) 2024-09-13T07:12:50.138 INFO:teuthology.orchestra.run.smithi031.stdout:1672 trunc from 0xd659221 to 0x5084fa6 2024-09-13T07:12:50.140 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:50.141+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:50.149 INFO:teuthology.orchestra.run.smithi031.stdout:1673 read 0x10846c9 thru 0x108a5e8 (0x5f20 bytes) 2024-09-13T07:12:50.149 INFO:teuthology.orchestra.run.smithi031.stdout:1674 read 0x3ef5315 thru 0x3f019bc (0xc6a8 bytes) 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4257.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:50.161 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 29.566895549000002, 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036526549999999999, 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594198+0000", 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594197+0000", 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594211+0000", 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.399e-05 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594220+0000", 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2359999999999993e-06 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594231+0000", 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0293e-05 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594280+0000", 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9261000000000002e-05 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.594301+0000", 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0744999999999999e-05 2024-09-13T07:12:50.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597812+0000", 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035110950000000001 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:20.597850+0000", 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8696999999999999e-05 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.165 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:908 3.10 3:09fecc17:::rbd_data.1173f77ecad8.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.916845138999999, 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040361349999999997, 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 908 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244248+0000", 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244247+0000", 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244253+0000", 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6389999999999997e-06 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244264+0000", 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1544e-05 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244270+0000", 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5790000000000003e-06 2024-09-13T07:12:50.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244309+0000", 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8838999999999999e-05 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244385+0000", 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6097e-05 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.244591+0000", 2024-09-13T07:12:50.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000206749 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246079+0000", 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0014876679999999999 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248230+0000", 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021511780000000001 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248264+0000", 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3283e-05 2024-09-13T07:12:50.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.248284+0000", 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0598000000000002e-05 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:913 3.1e e25/19 3:7c3d5185:::rbd_data.1173f77ecad8.0000000000000005:head v 25'4, mlcod=24'2)", 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.914486909000001, 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004381462, 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.171 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246606+0000", 2024-09-13T07:12:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246612+0000", 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9499999999999998e-06 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246622+0000", 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0217000000000001e-05 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246627+0000", 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0540000000000002e-06 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5330000000000002e-05 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246684+0000", 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1645e-05 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.250954+0000", 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042697819999999997 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.250988+0000", 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4215999999999999e-05 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:922 3.a 3:52856a69:::rbd_data.1173f77ecad8.000000000000000f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.909721393, 2024-09-13T07:12:50.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058310970000000004, 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 922 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251372+0000", 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251371+0000", 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251376+0000", 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9450000000000001e-06 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251387+0000", 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0786e-05 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251392+0000", 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3310000000000003e-06 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251426+0000", 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3767999999999999e-05 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.251503+0000", 2024-09-13T07:12:50.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7410000000000006e-05 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.257203+0000", 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056996900000000003 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1010 3.10 3:0d2ccec7:::rbd_data.1173f77ecad8.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.891867971, 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042914140000000003, 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.179 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1010 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269225+0000", 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269224+0000", 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269230+0000", 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7030000000000003e-06 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269243+0000", 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3215e-05 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269248+0000", 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0889999999999999e-06 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269292+0000", 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3928999999999999e-05 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269370+0000", 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8163000000000004e-05 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.269581+0000", 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021031700000000001 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.270174+0000", 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059290199999999997 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273429+0000", 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00325524 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273485+0000", 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5926e-05 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.273517+0000", 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1826000000000003e-05 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1052 3.10 3:0982ea78:::rbd_data.1173f77ecad8.000000000000003e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.878881171, 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052248729999999997, 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1052 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:50.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282212+0000", 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282211+0000", 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282217+0000", 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6350000000000003e-06 2024-09-13T07:12:50.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282228+0000", 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1323e-05 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282245+0000", 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6807e-05 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.187 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282279+0000", 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4100000000000002e-05 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.282365+0000", 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6012000000000003e-05 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287437+0000", 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050720519999999996 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.188 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1061 3.10 3:0c912c88:::rbd_data.1173f77ecad8.0000000000000044:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.875970625000001, 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005098455, 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1061 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.189 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285123+0000", 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285122+0000", 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.190 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285126+0000", 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3340000000000003e-06 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285135+0000", 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9039999999999998e-06 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285142+0000", 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8020000000000003e-06 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.191 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285181+0000", 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8717000000000002e-05 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.285304+0000", 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012349400000000001 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290221+0000", 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049169610000000001 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.192 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1067 3.10 3:0982ea78:::rbd_data.1173f77ecad8.000000000000003e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.873507231, 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035721809999999998, 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1067 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.193 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287586+0000", 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287585+0000", 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.194 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287590+0000", 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2560000000000004e-06 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287599+0000", 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3349999999999996e-06 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287604+0000", 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5470000000000004e-06 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.195 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287646+0000", 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1992e-05 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.287714+0000", 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7594000000000004e-05 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.288113+0000", 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00039889400000000001 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.196 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.288598+0000", 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000484981 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291095+0000", 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024973759999999999 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291137+0000", 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1987999999999998e-05 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.197 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.291158+0000", 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0919000000000001e-05 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1073 3.10 3:09473a54:::rbd_data.1173f77ecad8.0000000000000049:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.870714467999999, 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004417575, 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1073 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290379+0000", 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.199 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290378+0000", 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290383+0000", 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3179999999999997e-06 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290396+0000", 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2492e-05 2024-09-13T07:12:50.200 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290402+0000", 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.759e-06 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290439+0000", 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6915000000000001e-05 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290530+0000", 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0853999999999994e-05 2024-09-13T07:12:50.201 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.294796+0000", 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042662339999999998 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1074 3.10 3:0c912c88:::rbd_data.1173f77ecad8.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.870407839, 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055866580000000004, 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.202 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1074 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290685+0000", 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290691+0000", 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0800000000000002e-06 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290699+0000", 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7390000000000005e-06 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290704+0000", 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7280000000000003e-06 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290731+0000", 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7299000000000001e-05 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290776+0000", 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4848000000000002e-05 2024-09-13T07:12:50.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.290902+0000", 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012560100000000001 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.294803+0000", 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039015579999999998 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296179+0000", 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013752879999999999 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296230+0000", 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1270999999999998e-05 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.296272+0000", 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2035e-05 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1125 3.a 3:5135e61f:::rbd_data.1173f77ecad8.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:50.207 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.849576846, 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00420258, 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1125 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311516+0000", 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311520+0000", 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8919999999999999e-06 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311531+0000", 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0805000000000001e-05 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311538+0000", 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1400000000000002e-06 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311577+0000", 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8395000000000001e-05 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311664+0000", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7609999999999996e-05 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.311792+0000", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012732499999999999 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315178+0000", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033866970000000001 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315667+0000", 2024-09-13T07:12:50.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00048881399999999996 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315698+0000", 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0321999999999999e-05 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.315719+0000", 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1369000000000002e-05 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1130 3.1e e25/19 3:7b48162d:::rbd_data.1173f77ecad8.000000000000005c:head v 25'13, mlcod=25'10)", 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.848341838, 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005273247, 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312751+0000", 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312757+0000", 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9410000000000004e-06 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312769+0000", 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2799999999999999e-05 2024-09-13T07:12:50.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312776+0000", 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8199999999999999e-06 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312819+0000", 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2293999999999999e-05 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.312838+0000", 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9123e-05 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.317986+0000", 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051480340000000001 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.318025+0000", 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8914e-05 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.216 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1348 3.10 3:0e76b587:::rbd_data.1173c53667c.0000000000000057:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.644055108, 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038489119999999999, 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1348 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:50.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517038+0000", 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517036+0000", 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517046+0000", 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7370000000000001e-06 2024-09-13T07:12:50.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517066+0000", 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9658999999999999e-05 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517076+0000", 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0217999999999999e-05 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517134+0000", 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7779999999999999e-05 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517218+0000", 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4738000000000001e-05 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.517335+0000", 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000116573 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.519614+0000", 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022788830000000002 2024-09-13T07:12:50.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520800+0000", 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001186063 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520846+0000", 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6161000000000001e-05 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.520887+0000", 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0960000000000001e-05 2024-09-13T07:12:50.221 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1536 3.a 3:5448ec73:::rbd_data.1173c53667c.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.395576046, 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004075932, 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.222 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1536 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765517+0000", 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765516+0000", 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765521+0000", 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0499999999999999e-06 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765532+0000", 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1138000000000001e-05 2024-09-13T07:12:50.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765538+0000", 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765575+0000", 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7283000000000002e-05 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765630+0000", 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4818999999999998e-05 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.765695+0000", 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5215000000000005e-05 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.766354+0000", 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00065942400000000003 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769502+0000", 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031472549999999998 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769551+0000", 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9577999999999999e-05 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769593+0000", 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1983000000000001e-05 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1542 3.a 3:52be2a25:::rbd_data.1173c53667c.0000000000000043:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.391390305, 2024-09-13T07:12:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005091641, 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1542 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769703+0000", 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769702+0000", 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769708+0000", 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3509999999999998e-06 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769722+0000", 2024-09-13T07:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4334e-05 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769730+0000", 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7330000000000003e-06 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769789+0000", 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9077999999999999e-05 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769860+0000", 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0832000000000002e-05 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.769978+0000", 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000118257 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.770500+0000", 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00052145399999999997 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774714+0000", 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042146079999999999 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774760+0000", 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.583e-05 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.774795+0000", 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4288000000000001e-05 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.123900894, 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:12:50.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:12:50.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:12:50.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1768 3.a e25/19 ondisk, result = 0)", 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.109027809000001, 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010131984, 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:50.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052064+0000", 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052072+0000", 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9529999999999999e-06 2024-09-13T07:12:50.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052090+0000", 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8263000000000001e-05 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052102+0000", 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2078e-05 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062047+0000", 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0099446580000000003 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062059+0000", 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2455e-05 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062197+0000", 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013813200000000001 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.241 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3345 3.10 3:0be6394f:::rbd_data.11734c6039c3.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.8749797089999998, 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035746290000000002, 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3345 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286113+0000", 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286111+0000", 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999971 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286123+0000", 2024-09-13T07:12:50.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2471e-05 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286146+0000", 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3136999999999999e-05 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286156+0000", 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4550000000000001e-06 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286216+0000", 2024-09-13T07:12:50.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0245999999999999e-05 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.286223+0000", 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1590000000000004e-06 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288195+0000", 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001972279 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288274+0000", 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8882999999999997e-05 2024-09-13T07:12:50.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.288352+0000", 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8158999999999995e-05 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289021+0000", 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00066851300000000003 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289615+0000", 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059398999999999997 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289657+0000", 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1989000000000001e-05 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.289688+0000", 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1257000000000002e-05 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4259 3.10 3:0a0f6fac:::rbd_data.11736f46b2f8.0000000000000082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.938427+0000", 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 7.2226657269999999, 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035730979999999998, 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4259 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938427+0000", 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938427+0000", 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938425+0000", 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938436+0000", 2024-09-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0910999999999999e-05 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938456+0000", 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0041e-05 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938464+0000", 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6660000000000008e-06 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938508+0000", 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4554000000000001e-05 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938561+0000", 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2615999999999999e-05 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.938620+0000", 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9178999999999997e-05 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.939277+0000", 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00065729600000000003 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.941923+0000", 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026458879999999999 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.941966+0000", 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2899999999999999e-05 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.942001+0000", 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.455e-05 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:7141 3.1e e33/19 3:7c40247c:::rbd_data.1173b1ea0ed4.0000000000000007:head v 33'63, mlcod=33'61)", 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:48.705519+0000", 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.455574661, 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003557798, 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.705519+0000", 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.705519+0000", 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.705518+0000", 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.707555+0000", 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002037339 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.707573+0000", 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7895000000000001e-05 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.707582+0000", 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1169999999999993e-06 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.707636+0000", 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4079999999999997e-05 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.707654+0000", 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7592e-05 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.709032+0000", 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001378818 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:48.709076+0000", 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3948e-05 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:50.259 INFO:teuthology.orchestra.run.smithi031.stdout:1675 writesame 0xcd3002d thru 0xcd368e7 data_size 0x1b(0x68bb bytes) 2024-09-13T07:12:50.259 INFO:teuthology.orchestra.run.smithi031.stdout:1676 write 0x77c9d53 thru 0x77d62d1 (0xc57f bytes) 2024-09-13T07:12:50.259 INFO:teuthology.orchestra.run.smithi031.stdout:1677 read 0x43fc5e9 thru 0x440acfd (0xe715 bytes) 2024-09-13T07:12:50.259 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.259 INFO:teuthology.orchestra.run.smithi031.stdout:1679 trunc from 0xcd368e8 to 0x5e7251 2024-09-13T07:12:50.259 INFO:teuthology.orchestra.run.smithi031.stdout:1682 clone 9 order 19 su 131072 sc 3 2024-09-13T07:12:50.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:50.260 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:12:50.261 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:50.259+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:50.302 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone8 from 0x5e7251 (overlap 0x5e7251) to 0x2f3928 2024-09-13T07:12:50.320 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.16.7 2024-09-13T07:12:50.353 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:12:50.358 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:12:50.358 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:12:50.364 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:50.365+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:50.375 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T07:12:50.376 INFO:teuthology.orchestra.run.smithi031.stdout:1684 punch from 0x53072c to 0x53d2cd, (0xcba1 bytes) 2024-09-13T07:12:50.388 INFO:teuthology.orchestra.run.smithi031.stdout:1685 write 0x2bd98ea thru 0x2be1f32 (0x8649 bytes) 2024-09-13T07:12:50.391 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.403 INFO:teuthology.orchestra.run.smithi031.stdout:1687 write 0x8caf4ce thru 0x8cb2781 (0x32b4 bytes) 2024-09-13T07:12:50.406 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.406 INFO:teuthology.orchestra.run.smithi031.stdout:1689 read 0x7c22258 thru 0x7c2a0d1 (0x7e7a bytes) 2024-09-13T07:12:50.415 INFO:teuthology.orchestra.run.smithi031.stdout:1690 trunc from 0x8cb2782 to 0xe817e1 2024-09-13T07:12:50.423 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:50.423 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:50.430 INFO:teuthology.orchestra.run.smithi031.stdout:1693 write 0x201975f thru 0x201a0c2 (0x964 bytes) 2024-09-13T07:12:50.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:50.452 INFO:teuthology.orchestra.run.smithi031.stdout:1694 write 0xdb3190c thru 0xdb33abc (0x21b1 bytes) 2024-09-13T07:12:50.456 INFO:teuthology.orchestra.run.smithi031.stdout:1695 trunc from 0xdb33abd to 0xd2417ea 2024-09-13T07:12:50.460 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:50.460+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:50.467 INFO:teuthology.orchestra.run.smithi031.stdout:1696 write 0xbbef107 thru 0xbbef306 (0x200 bytes) 2024-09-13T07:12:50.471 INFO:teuthology.orchestra.run.smithi031.stdout:1698 read 0xc878cef thru 0xc87e2ce (0x55e0 bytes) 2024-09-13T07:12:50.472 INFO:teuthology.orchestra.run.smithi031.stdout:1701 trunc from 0xd2417ea to 0xb710a59 2024-09-13T07:12:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:1702 write 0xe548e9d thru 0xe55529d (0xc401 bytes) 2024-09-13T07:12:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:1704 write 0x4a4c79d thru 0x4a55004 (0x8868 bytes) 2024-09-13T07:12:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:1705 punch from 0x929651f to 0x9296560, (0x41 bytes) 2024-09-13T07:12:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:1707 read 0x10c6a13 thru 0x10c9583 (0x2b71 bytes) 2024-09-13T07:12:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:1708 trunc from 0xe55529e to 0xcc8fd9b 2024-09-13T07:12:50.506 INFO:teuthology.orchestra.run.smithi031.stdout:1709 read 0xa35604b thru 0xa35bda8 (0x5d5e bytes) 2024-09-13T07:12:50.506 INFO:teuthology.orchestra.run.smithi031.stdout:1710 writesame 0x78af2ee thru 0x78b04e2 data_size 0x1(0x11f5 bytes) 2024-09-13T07:12:50.510 INFO:teuthology.orchestra.run.smithi031.stdout:1712 punch from 0xc1db571 to 0xc1e7228, (0xbcb7 bytes) 2024-09-13T07:12:50.510 INFO:teuthology.orchestra.run.smithi031.stdout:1713 read 0x161c12d thru 0x162b69a (0xf56e bytes) 2024-09-13T07:12:50.510 INFO:teuthology.orchestra.run.smithi031.stdout:1715 read 0x8b25363 thru 0x8b289d7 (0x3675 bytes) 2024-09-13T07:12:50.510 INFO:teuthology.orchestra.run.smithi031.stdout:1716 writesame 0xc1f4718 thru 0xc1fad35 data_size 0x3(0x661e bytes) 2024-09-13T07:12:50.514 INFO:teuthology.orchestra.run.smithi031.stdout:1717 read 0x6f15e68 thru 0x6f1e2a6 (0x843f bytes) 2024-09-13T07:12:50.515 INFO:teuthology.orchestra.run.smithi031.stdout:1718 write 0x5858a4d thru 0x5858d05 (0x2b9 bytes) 2024-09-13T07:12:50.523 INFO:teuthology.orchestra.run.smithi031.stdout:1719 write 0xd7b3174 thru 0xd7b9a94 (0x6921 bytes) 2024-09-13T07:12:50.526 INFO:teuthology.orchestra.run.smithi031.stdout:1723 read 0x956c751 thru 0x957b685 (0xef35 bytes) 2024-09-13T07:12:50.526 INFO:teuthology.orchestra.run.smithi031.stdout:1725 read 0xaebdf9a thru 0xaebefe6 (0x104d bytes) 2024-09-13T07:12:50.526 INFO:teuthology.orchestra.run.smithi031.stdout:1727 write 0x84bdab1 thru 0x84c9398 (0xb8e8 bytes) 2024-09-13T07:12:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:1728 read 0x7b5bf37 thru 0x7b5d3b9 (0x1483 bytes) 2024-09-13T07:12:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:1731 write 0xb7ef7e1 thru 0xb7fce17 (0xd637 bytes) 2024-09-13T07:12:50.536 INFO:teuthology.orchestra.run.smithi031.stdout:1734 write 0x19c99a6 thru 0x19cd273 (0x38ce bytes) 2024-09-13T07:12:50.538 INFO:teuthology.orchestra.run.smithi031.stdout:1738 read 0x98c781b thru 0x98d374f (0xbf35 bytes) 2024-09-13T07:12:50.544 INFO:teuthology.orchestra.run.smithi031.stdout:1740 trunc from 0xd7b9a95 to 0x8d8a707 2024-09-13T07:12:50.554 INFO:teuthology.orchestra.run.smithi031.stdout:1741 write 0x1f2506e thru 0x1f2c48c (0x741f bytes) 2024-09-13T07:12:50.557 INFO:teuthology.orchestra.run.smithi031.stdout:1742 punch from 0x1455df2 to 0x1457d8b, (0x1f99 bytes) 2024-09-13T07:12:50.561 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:50.562+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:50.566 INFO:teuthology.orchestra.run.smithi031.stdout:1744 write 0xc7e9c0d thru 0xc7ea47f (0x873 bytes) 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout:1745 read 0xa5aa036 thru 0xa5ac1c1 (0x218c bytes) 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout:1748 write 0x9ca9ed9 thru 0x9cad268 (0x3390 bytes) 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:50.569 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:50.572 INFO:teuthology.orchestra.run.smithi031.stdout:1749 write 0x4c1f90d thru 0x4c2f441 (0xfb35 bytes) 2024-09-13T07:12:50.575 INFO:teuthology.orchestra.run.smithi031.stdout:1750 read 0x9c91107 thru 0x9c962cb (0x51c5 bytes) 2024-09-13T07:12:50.575 INFO:teuthology.orchestra.run.smithi031.stdout:1752 read 0x7e5f67e thru 0x7e63382 (0x3d05 bytes) 2024-09-13T07:12:50.575 INFO:teuthology.orchestra.run.smithi031.stdout:1757 write 0xb0e9b51 thru 0xb0f5e49 (0xc2f9 bytes) 2024-09-13T07:12:50.578 INFO:teuthology.orchestra.run.smithi031.stdout:1758 read 0x3128adf thru 0x31338de (0xae00 bytes) 2024-09-13T07:12:50.578 INFO:teuthology.orchestra.run.smithi031.stdout:1759 write 0x801cf10 thru 0x8026874 (0x9965 bytes) 2024-09-13T07:12:50.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:1760 writesame 0xd05290c thru 0xd0575a6 data_size 0x3(0x4c9b bytes) 2024-09-13T07:12:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:1762 punch from 0x2efdc5b to 0x2f0d9b4, (0xfd59 bytes) 2024-09-13T07:12:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:1765 write 0xb581806 thru 0xb587aec (0x62e7 bytes) 2024-09-13T07:12:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:1768 read 0xae729b thru 0xaf607b (0xede1 bytes) 2024-09-13T07:12:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:1770 writesame 0x6414a09 thru 0x6422ce0 data_size 0x44(0xe2d8 bytes) 2024-09-13T07:12:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:1771 read 0xae1126a thru 0xae1d5f0 (0xc387 bytes) 2024-09-13T07:12:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:1774 write 0x506378d thru 0x506dec4 (0xa738 bytes) 2024-09-13T07:12:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:1776 read 0x9edae36 thru 0x9eea008 (0xf1d3 bytes) 2024-09-13T07:12:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:1777 punch from 0x6085951 to 0x609163b, (0xbcea bytes) 2024-09-13T07:12:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:1778 read 0x9d9a867 thru 0x9d9db35 (0x32cf bytes) 2024-09-13T07:12:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:1779 write 0xa076a8d thru 0xa07be1a (0x538e bytes) 2024-09-13T07:12:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:1780 punch from 0x15f809 to 0x16c280, (0xca77 bytes) 2024-09-13T07:12:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:1782 writesame 0x601d85c thru 0x6028b67 data_size 0xe(0xb30c bytes) 2024-09-13T07:12:50.607 INFO:teuthology.orchestra.run.smithi031.stdout:1783 write 0x67364d9 thru 0x673929b (0x2dc3 bytes) 2024-09-13T07:12:50.609 INFO:teuthology.orchestra.run.smithi031.stdout:1784 punch from 0x699104d to 0x6995011, (0x3fc4 bytes) 2024-09-13T07:12:50.610 INFO:teuthology.orchestra.run.smithi031.stdout:1785 writesame 0x52dad9b thru 0x52e6c1b data_size 0x1(0xbe81 bytes) 2024-09-13T07:12:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:1787 read 0x6e0b58 thru 0x6e1c59 (0x1102 bytes) 2024-09-13T07:12:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:1788 write 0x41f1c79 thru 0x4200280 (0xe608 bytes) 2024-09-13T07:12:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:1789 write 0x8971fb3 thru 0x897a29e (0x82ec bytes) 2024-09-13T07:12:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:1790 read 0x18920d8 thru 0x1897271 (0x519a bytes) 2024-09-13T07:12:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:1791 read 0xa126451 thru 0xa12ca6a (0x661a bytes) 2024-09-13T07:12:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:1793 read 0x8e0d59c thru 0x8e17880 (0xa2e5 bytes) 2024-09-13T07:12:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:1794 write 0x4f53530 thru 0x4f63274 (0xfd45 bytes) 2024-09-13T07:12:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:1798 trunc from 0xd0575a7 to 0x58c3406 2024-09-13T07:12:50.662 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:50.662+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:50.666 INFO:teuthology.orchestra.run.smithi031.stdout:1799 trunc from 0x58c3406 to 0xa248844 2024-09-13T07:12:50.672 INFO:teuthology.orchestra.run.smithi031.stdout:1801 read 0x3344b05 thru 0x334964a (0x4b46 bytes) 2024-09-13T07:12:50.673 INFO:teuthology.orchestra.run.smithi031.stdout:1802 punch from 0x2812506 to 0x2818efd, (0x69f7 bytes) 2024-09-13T07:12:50.673 INFO:teuthology.orchestra.run.smithi031.stdout:1803 write 0x52c0364 thru 0x52cfbaf (0xf84c bytes) 2024-09-13T07:12:50.683 INFO:teuthology.orchestra.run.smithi031.stdout:1804 writesame 0xce0cb3d thru 0xce17887 data_size 0x25(0xad4b bytes) 2024-09-13T07:12:50.683 INFO:teuthology.orchestra.run.smithi031.stdout:1805 writesame 0xcb31db4 thru 0xcb359cc data_size 0x11(0x3c19 bytes) 2024-09-13T07:12:50.686 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.686 INFO:teuthology.orchestra.run.smithi031.stdout:1807 read 0x7ce6154 thru 0x7cec13d (0x5fea bytes) 2024-09-13T07:12:50.686 INFO:teuthology.orchestra.run.smithi031.stdout:1808 read 0xaf2bb1e thru 0xaf3729a (0xb77d bytes) 2024-09-13T07:12:50.686 INFO:teuthology.orchestra.run.smithi031.stdout:1809 read 0xce16c82 thru 0xce17887 (0xc06 bytes) 2024-09-13T07:12:50.701 INFO:teuthology.orchestra.run.smithi031.stdout:1810 trunc from 0xce17888 to 0x4a16f7 2024-09-13T07:12:50.711 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:50.711 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:50.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:906 3.1 3:80f3f876:::rbd_data.1173f77ecad8.0000000000000009:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.242359+0000", 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.468640266, 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061750620000000003, 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 906 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242359+0000", 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242359+0000", 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242358+0000", 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.242364+0000", 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4389999999999997e-06 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243101+0000", 2024-09-13T07:12:50.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073671900000000005 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243111+0000", 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7909999999999999e-06 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243405+0000", 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029383000000000002 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.243526+0000", 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121458 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248534+0000", 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050079510000000001 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:916 3.12 3:4ade7e99:::rbd_data.1173f77ecad8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.464337357, 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081196749999999998, 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 916 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:50.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246662+0000", 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246661+0000", 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246666+0000", 2024-09-13T07:12:50.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4450000000000004e-06 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246681+0000", 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5104e-05 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246689+0000", 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246739+0000", 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9855000000000003e-05 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.246823+0000", 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3820999999999996e-05 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.247040+0000", 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021744099999999999 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.247583+0000", 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054293699999999996 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254695+0000", 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071119429999999999 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254741+0000", 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5507999999999999e-05 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254782+0000", 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1047999999999997e-05 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:920 3.1 3:87900178:::rbd_data.1173f77ecad8.000000000000000e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.462469941, 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060296539999999997, 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 920 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:50.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248529+0000", 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248534+0000", 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-13T07:12:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248544+0000", 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7210000000000006e-06 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248550+0000", 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0589999999999999e-06 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248587+0000", 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6783999999999999e-05 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.248704+0000", 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117199 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254559+0000", 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058553570000000003 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:921 3.1 3:80f3f876:::rbd_data.1173f77ecad8.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.249040+0000", 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.461959844000001, 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005632448, 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 921 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249040+0000", 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249040+0000", 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249039+0000", 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249045+0000", 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249059+0000", 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3925e-05 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249066+0000", 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249114+0000", 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8628000000000003e-05 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249198+0000", 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3560000000000006e-05 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249396+0000", 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019819600000000001 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.249786+0000", 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039016700000000002 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254602+0000", 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004815204 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254639+0000", 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7107999999999998e-05 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.254672+0000", 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3246e-05 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1015 3.6 3:65878748:::rbd_data.1173f77ecad8.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.270768+0000", 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.440231051, 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059243009999999999, 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1015 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270768+0000", 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270768+0000", 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270767+0000", 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270773+0000", 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270783+0000", 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9240000000000002e-06 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270789+0000", 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270831+0000", 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1709000000000003e-05 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.270927+0000", 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6464000000000003e-05 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:50.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.271210+0000", 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028254799999999998 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276509+0000", 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005298929 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276607+0000", 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8502e-05 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276656+0000", 2024-09-13T07:12:50.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8566000000000002e-05 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.276693+0000", 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6808999999999999e-05 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.3 epoch 25/19 rep_tid 30 entries 25'11 (0'0) error 3:c2b5a8ac:::rbd_data.1173f77ecad8.0000000000000061:head by client.4467.0:1128 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'8)", 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.312215+0000", 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.398784799, 2024-09-13T07:12:50.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056486449999999999, 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312215+0000", 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312215+0000", 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312214+0000", 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312221+0000", 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.973e-06 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312234+0000", 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3406e-05 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312242+0000", 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.312314+0000", 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2157999999999996e-05 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.317863+0000", 2024-09-13T07:12:50.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055494949999999998 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1153 3.1b 3:d88532a6:::rbd_data.1173f77ecad8.000000000000015c:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.384104067999999, 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010064101000000001, 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1153 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326895+0000", 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326898+0000", 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2660000000000002e-06 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326906+0000", 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4789999999999999e-06 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326910+0000", 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.326959+0000", 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8327e-05 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327025+0000", 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6161999999999994e-05 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.327352+0000", 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032693799999999998 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.330855+0000", 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003503501 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.336879+0000", 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060237880000000004 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.336918+0000", 2024-09-13T07:12:50.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9400999999999998e-05 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.336959+0000", 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0972e-05 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1317 3.17 3:ec069b17:::rbd_data.1173c53667c.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.238824230000001, 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057872000000000002, 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1317 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472175+0000", 2024-09-13T07:12:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472174+0000", 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472180+0000", 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2859999999999996e-06 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472193+0000", 2024-09-13T07:12:50.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2581e-05 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472200+0000", 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5349999999999999e-06 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472255+0000", 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4957999999999998e-05 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472395+0000", 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014002400000000001 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472584+0000", 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018895399999999999 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475965+0000", 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033809349999999998 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.477910+0000", 2024-09-13T07:12:50.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019448849999999999 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.477941+0000", 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0874999999999998e-05 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.477962+0000", 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1268999999999999e-05 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1344 3.6 3:65d64014:::rbd_data.1173c53667c.0000000000000053:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.196829218, 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064892170000000002, 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1344 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514170+0000", 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514174+0000", 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514182+0000", 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0930000000000003e-06 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514189+0000", 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-09-13T07:12:50.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.514715+0000", 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052648699999999996 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.515279+0000", 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056402199999999996 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.515416+0000", 2024-09-13T07:12:50.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013629000000000001 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.518278+0000", 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028619800000000001 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.520605+0000", 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023270019999999999 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.520639+0000", 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3970000000000002e-05 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.520659+0000", 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0684000000000001e-05 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1631 3.17 3:efa3965d:::rbd_data.1173c53667c.00000000000000aa:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.867573+0000", 2024-09-13T07:12:50.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.843426768, 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020209943000000001, 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1631 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867573+0000", 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867573+0000", 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867571+0000", 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867578+0000", 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6379999999999998e-06 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867590+0000", 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2086000000000001e-05 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867596+0000", 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867656+0000", 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0504999999999998e-05 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.867764+0000", 2024-09-13T07:12:50.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107629 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.871357+0000", 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003592947 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887667+0000", 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016309939999999998 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887716+0000", 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9199000000000001e-05 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887754+0000", 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8519000000000002e-05 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887783+0000", 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8160000000000001e-05 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:1631 3.17 e25/19 ondisk, result = 0)", 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.829492874, 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006282429, 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:50.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881506+0000", 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881510+0000", 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4649999999999996e-06 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881523+0000", 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2604e-05 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.881529+0000", 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5520000000000001e-06 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887704+0000", 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061750750000000004 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887711+0000", 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4980000000000003e-06 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.887789+0000", 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8014000000000002e-05 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1640 3.1 3:87b03873:::rbd_data.1173c53667c.00000000000000a0:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:50.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.813788934, 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015780835999999999, 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1640 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:50.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897210+0000", 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897214+0000", 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.617e-06 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897225+0000", 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0747000000000001e-05 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897230+0000", 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9960000000000001e-06 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897280+0000", 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9514000000000001e-05 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.897413+0000", 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133304 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.899854+0000", 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024406950000000001 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912923+0000", 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013069173999999999 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912931+0000", 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6950000000000006e-06 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912965+0000", 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.396e-05 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.912991+0000", 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5856000000000001e-05 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1828 3.18 e26/19 3:1c28841c:::rbd_data.1173c53667c.00000000000000b8:head v 26'17, mlcod=25'15)", 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.561964061999999, 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057524409999999996, 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149035+0000", 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149056+0000", 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1333000000000001e-05 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149065+0000", 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9160000000000007e-06 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149074+0000", 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.541e-06 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149115+0000", 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1014999999999999e-05 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.149135+0000", 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0435999999999999e-05 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154761+0000", 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056255630000000001 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154788+0000", 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.688e-05 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:2858 3.1b 3:d93b9695:::rbd_data.11734c6039c3.000000000000006c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.617151+0000", 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.093848287, 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081434639999999996, 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2858 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617151+0000", 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617151+0000", 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617149+0000", 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617160+0000", 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.128e-05 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617178+0000", 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.73e-05 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617184+0000", 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8260000000000004e-06 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617233+0000", 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8573999999999999e-05 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.617366+0000", 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000132884 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.618603+0000", 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012372139999999999 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625232+0000", 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066293020000000001 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625240+0000", 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2860000000000002e-06 2024-09-13T07:12:50.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625270+0000", 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0134999999999999e-05 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.625295+0000", 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4683999999999999e-05 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3319 3.17 3:ebf18aab:::rbd_data.11734c6039c3.0000000000000030:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.228186+0000", 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4828135870000008, 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086840040000000004, 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3319 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228186+0000", 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228186+0000", 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228184+0000", 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228193+0000", 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1649999999999995e-06 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228210+0000", 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7116999999999999e-05 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228217+0000", 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9999999999999999e-06 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228260+0000", 2024-09-13T07:12:50.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2342000000000002e-05 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.228266+0000", 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0059999999999998e-06 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.229955+0000", 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001689308 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.230025+0000", 2024-09-13T07:12:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9962999999999994e-05 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.231187+0000", 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011621730000000001 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236776+0000", 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055889290000000003 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236820+0000", 2024-09-13T07:12:50.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3544000000000002e-05 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236851+0000", 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1420000000000001e-05 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.236870+0000", 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8689999999999999e-05 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4054 3.6 3:60dce801:::rbd_data.11736f46b2f8.000000000000011a:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:50.787 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.727538+0000", 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9834618449999999, 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080239769999999998, 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4054 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727538+0000", 2024-09-13T07:12:50.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727538+0000", 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727536+0000", 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727544+0000", 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3710000000000002e-06 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727561+0000", 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6979000000000001e-05 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727569+0000", 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4479999999999994e-06 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727621+0000", 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1413999999999999e-05 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.727723+0000", 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102645 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.729014+0000", 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012910549999999999 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735413+0000", 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063984469999999998 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735460+0000", 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7068999999999999e-05 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735519+0000", 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9487999999999997e-05 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.735561+0000", 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2222000000000003e-05 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4076 3.12 3:4f45d645:::rbd_data.11736f46b2f8.000000000000018e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.761011+0000", 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9499882980000001, 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056740929999999998, 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4076 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761011+0000", 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761011+0000", 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761010+0000", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761015+0000", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6920000000000001e-06 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761026+0000", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1003e-05 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761031+0000", 2024-09-13T07:12:50.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4729999999999999e-06 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761065+0000", 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4010000000000001e-05 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761112+0000", 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6999000000000002e-05 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761172+0000", 2024-09-13T07:12:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9312999999999999e-05 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.761581+0000", 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000409116 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.766635+0000", 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050541370000000002 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.766662+0000", 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7069000000000001e-05 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.766685+0000", 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3045999999999999e-05 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7044 3.3 e33/19 3:c3722553:::rbd_data.1173b1ea0ed4.0000000000000008:head v 33'34, mlcod=29'32)", 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.691231+0000", 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.0197687370000001, 2024-09-13T07:12:50.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065008410000000003, 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691231+0000", 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691231+0000", 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691230+0000", 2024-09-13T07:12:50.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691239+0000", 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2070000000000001e-06 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691248+0000", 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691254+0000", 2024-09-13T07:12:50.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9510000000000003e-06 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691503+0000", 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024897800000000001 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.691521+0000", 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8465999999999999e-05 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.697678+0000", 2024-09-13T07:12:50.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061561660000000002 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.697731+0000", 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3801999999999998e-05 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7116 3.12 3:4b104053:::rbd_data.11736f46b2f8.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693961+0000", 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.0170381580000001, 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005586756, 2024-09-13T07:12:50.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7116 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693961+0000", 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693961+0000", 2024-09-13T07:12:50.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693961+0000", 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693963+0000", 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0839999999999999e-06 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693970+0000", 2024-09-13T07:12:50.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693976+0000", 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3289999999999999e-06 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.699373+0000", 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053966099999999996 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.699445+0000", 2024-09-13T07:12:50.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2416000000000006e-05 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.699548+0000", 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102876 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7136 3.1f 3:fd178933:::rbd_data.1173b1ea0ed4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.699180+0000", 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.011819129, 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057478290000000003, 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:50.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7136 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.699180+0000", 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.699180+0000", 2024-09-13T07:12:50.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.699179+0000", 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.700525+0000", 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013457619999999999 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.700540+0000", 2024-09-13T07:12:50.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5756999999999998e-05 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.700550+0000", 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999993e-06 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.700596+0000", 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6887e-05 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.700711+0000", 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114922 2024-09-13T07:12:50.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701035+0000", 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032321599999999998 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.704115+0000", 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030800620000000002 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.704817+0000", 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070237100000000003 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.704862+0000", 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5179000000000002e-05 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.704928+0000", 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5825000000000001e-05 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:50.811 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:50.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:50.815 INFO:teuthology.orchestra.run.smithi031.stdout:1811 write 0x6e78082 thru 0x6e7b300 (0x327f bytes) 2024-09-13T07:12:50.815 INFO:teuthology.orchestra.run.smithi031.stdout:1812 write 0x4446f7f thru 0x4453cd2 (0xcd54 bytes) 2024-09-13T07:12:50.815 INFO:teuthology.orchestra.run.smithi031.stdout:1815 read 0x43734f thru 0x43f5e0 (0x8292 bytes) 2024-09-13T07:12:50.815 INFO:teuthology.orchestra.run.smithi031.stdout:1818 trunc from 0x6e7b301 to 0xb696aa1 2024-09-13T07:12:50.815 INFO:teuthology.orchestra.run.smithi031.stdout:1819 punch from 0xee1bd3 to 0xee1eb8, (0x2e5 bytes) 2024-09-13T07:12:50.815 INFO:teuthology.orchestra.run.smithi031.stdout:1820 write 0xc6e6a10 thru 0xc6e941c (0x2a0d bytes) 2024-09-13T07:12:50.815 INFO:teuthology.orchestra.run.smithi031.stdout:1821 writesame 0x7633ae6 thru 0x76389f9 data_size 0x2a(0x4f14 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1822 read 0x189a234 thru 0x18a1e9f (0x7c6c bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1824 punch from 0x9e1f448 to 0x9e20ee9, (0x1aa1 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1825 write 0x5e89c00 thru 0x5e94837 (0xac38 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1826 read 0x64349fc thru 0x6442cf4 (0xe2f9 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1829 write 0xe8f7563 thru 0xe8fdbf1 (0x668f bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1832 write 0x969300d thru 0x96a2217 (0xf20b bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1834 write 0x178d88a thru 0x179899d (0xb114 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1835 read 0x228ebea thru 0x2295d93 (0x71aa bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1837 read 0xc3c0ce0 thru 0xc3d09db (0xfcfc bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1838 write 0x4bee988 thru 0x4bf0957 (0x1fd0 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1841 write 0xa085e80 thru 0xa08c4ca (0x664b bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1842 read 0x4a357dc thru 0x4a3a405 (0x4c2a bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1843 write 0x7a64cda thru 0x7a6bbe0 (0x6f07 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1844 trunc from 0xe8fdbf2 to 0x748c32a 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1846 punch from 0x125cb49 to 0x126a479, (0xd930 bytes) 2024-09-13T07:12:50.816 INFO:teuthology.orchestra.run.smithi031.stdout:1847 punch from 0x4977617 to 0x4980519, (0x8f02 bytes) 2024-09-13T07:12:50.817 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:50.813+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:50.826 INFO:teuthology.orchestra.run.smithi031.stdout:1848 writesame 0xd9563c8 thru 0xd9581a6 data_size 0x1(0x1ddf bytes) 2024-09-13T07:12:50.838 INFO:teuthology.orchestra.run.smithi031.stdout:1850 trunc from 0xd9581a7 to 0x2a4a5bd 2024-09-13T07:12:50.859 INFO:teuthology.orchestra.run.smithi031.stdout:1851 write 0x6c7e0ce thru 0x6c8da06 (0xf939 bytes) 2024-09-13T07:12:50.865 INFO:teuthology.orchestra.run.smithi031.stdout:1852 write 0x4d070d3 thru 0x4d16c7e (0xfbac bytes) 2024-09-13T07:12:50.869 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.873 INFO:teuthology.orchestra.run.smithi031.stdout:1854 trunc from 0x6c8da07 to 0xa652ac5 2024-09-13T07:12:50.879 INFO:teuthology.orchestra.run.smithi031.stdout:1858 writesame 0x5f73350 thru 0x5f7dad6 data_size 0x1(0xa787 bytes) 2024-09-13T07:12:50.902 INFO:teuthology.orchestra.run.smithi031.stdout:1859 writesame 0x130ea87 thru 0x1317247 data_size 0x1(0x87c1 bytes) 2024-09-13T07:12:50.913 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:50.914+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:50.920 INFO:teuthology.orchestra.run.smithi031.stdout:1860 read 0x88843e0 thru 0x888519f (0xdc0 bytes) 2024-09-13T07:12:50.928 INFO:teuthology.orchestra.run.smithi031.stdout:1865 write 0xbfdea7b thru 0xbfe97f9 (0xad7f bytes) 2024-09-13T07:12:50.934 INFO:teuthology.orchestra.run.smithi031.stdout:1866 read 0xa533e97 thru 0xa53fb70 (0xbcda bytes) 2024-09-13T07:12:50.934 INFO:teuthology.orchestra.run.smithi031.stdout:1868 punch from 0x51f61fb to 0x51fb677, (0x547c bytes) 2024-09-13T07:12:50.934 INFO:teuthology.orchestra.run.smithi031.stdout:1870 read 0x2ce2907 thru 0x2ce52a3 (0x299d bytes) 2024-09-13T07:12:50.934 INFO:teuthology.orchestra.run.smithi031.stdout:1871 write 0x399beee thru 0x39a1ffc (0x610f bytes) 2024-09-13T07:12:50.946 INFO:teuthology.orchestra.run.smithi031.stdout:1872 writesame 0xe67bc53 thru 0xe6898e8 data_size 0x5(0xdc96 bytes) 2024-09-13T07:12:50.947 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:50.947 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:50.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:50.962 INFO:teuthology.orchestra.run.smithi031.stdout:1873 trunc from 0xe6898e9 to 0xb3d0df 2024-09-13T07:12:50.991 INFO:teuthology.orchestra.run.smithi031.stdout:1874 write 0xb39eb9e thru 0xb3adb3f (0xefa2 bytes) 2024-09-13T07:12:50.994 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.994 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:50.994 INFO:teuthology.orchestra.run.smithi031.stdout:1879 read 0x588dca5 thru 0x589d74b (0xfaa7 bytes) 2024-09-13T07:12:50.994 INFO:teuthology.orchestra.run.smithi031.stdout:1881 read 0x62d1999 thru 0x62d6c9e (0x5306 bytes) 2024-09-13T07:12:50.994 INFO:teuthology.orchestra.run.smithi031.stdout:1882 write 0x696a7f8 thru 0x697340a (0x8c13 bytes) 2024-09-13T07:12:50.998 INFO:teuthology.orchestra.run.smithi031.stdout:1883 trunc from 0xb3adb40 to 0x9fe4a10 2024-09-13T07:12:51.008 INFO:teuthology.orchestra.run.smithi031.stdout:1884 punch from 0x45b5ff0 to 0x45b847c, (0x248c bytes) 2024-09-13T07:12:51.008 INFO:teuthology.orchestra.run.smithi031.stdout:1886 read 0x1e73c48 thru 0x1e7f799 (0xbb52 bytes) 2024-09-13T07:12:51.008 INFO:teuthology.orchestra.run.smithi031.stdout:1887 punch from 0x861c00b to 0x86268a5, (0xa89a bytes) 2024-09-13T07:12:51.008 INFO:teuthology.orchestra.run.smithi031.stdout:1889 read 0x57a568c thru 0x57b4a00 (0xf375 bytes) 2024-09-13T07:12:51.008 INFO:teuthology.orchestra.run.smithi031.stdout:1890 writesame 0x901ee9c thru 0x9029158 data_size 0x9(0xa2bd bytes) 2024-09-13T07:12:51.013 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:51.014+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:51.015 INFO:teuthology.orchestra.run.smithi031.stdout:1891 read 0x15ea5dd thru 0x15f9e55 (0xf879 bytes) 2024-09-13T07:12:51.015 INFO:teuthology.orchestra.run.smithi031.stdout:1892 read 0x931ea30 thru 0x9326661 (0x7c32 bytes) 2024-09-13T07:12:51.015 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:1894 write 0xa7001b5 thru 0xa70720c (0x7058 bytes) 2024-09-13T07:12:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:1897 read 0xa24b846 thru 0xa24e9ac (0x3167 bytes) 2024-09-13T07:12:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:1898 read 0x49d4f39 thru 0x49d6bd4 (0x1c9c bytes) 2024-09-13T07:12:51.031 INFO:teuthology.orchestra.run.smithi031.stdout:1899 writesame 0xc9eea87 thru 0xc9f0968 data_size 0x1(0x1ee2 bytes) 2024-09-13T07:12:51.031 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.031 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.031 INFO:teuthology.orchestra.run.smithi031.stdout:1903 read 0xc148fb7 thru 0xc14958f (0x5d9 bytes) 2024-09-13T07:12:51.031 INFO:teuthology.orchestra.run.smithi031.stdout:1904 write 0xa928560 thru 0xa937acc (0xf56d bytes) 2024-09-13T07:12:51.037 INFO:teuthology.orchestra.run.smithi031.stdout:1905 trunc from 0xc9f0969 to 0xa11650c 2024-09-13T07:12:51.046 INFO:teuthology.orchestra.run.smithi031.stdout:1907 writesame 0x7edb4f1 thru 0x7edcf06 data_size 0x3f(0x1a16 bytes) 2024-09-13T07:12:51.049 INFO:teuthology.orchestra.run.smithi031.stdout:1909 writesame 0x3741df0 thru 0x3747697 data_size 0x4(0x58a8 bytes) 2024-09-13T07:12:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:1910 trunc from 0xa11650c to 0xbf10e31 2024-09-13T07:12:51.060 INFO:teuthology.orchestra.run.smithi031.stdout:1911 read 0x2adaa7c thru 0x2ae8b55 (0xe0da bytes) 2024-09-13T07:12:51.061 INFO:teuthology.orchestra.run.smithi031.stdout:1912 write 0x5bb36b3 thru 0x5bbe4d2 (0xae20 bytes) 2024-09-13T07:12:51.064 INFO:teuthology.orchestra.run.smithi031.stdout:1914 write 0xa2c9252 thru 0xa2d8279 (0xf028 bytes) 2024-09-13T07:12:51.067 INFO:teuthology.orchestra.run.smithi031.stdout:1915 write 0x4002ed1 thru 0x400bdda (0x8f0a bytes) 2024-09-13T07:12:51.082 INFO:teuthology.orchestra.run.smithi031.stdout:1916 trunc from 0xbf10e31 to 0x14fc9e7 2024-09-13T07:12:51.101 INFO:teuthology.orchestra.run.smithi031.stdout:1918 write 0x4d2ce22 thru 0x4d32056 (0x5235 bytes) 2024-09-13T07:12:51.110 INFO:teuthology.orchestra.run.smithi031.stdout:1919 writesame 0x5bf8b3d thru 0x5c03f82 data_size 0x82(0xb446 bytes) 2024-09-13T07:12:51.110 INFO:teuthology.orchestra.run.smithi031.stdout:1920 write 0x13f941f thru 0x13fad40 (0x1922 bytes) 2024-09-13T07:12:51.113 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.113 INFO:teuthology.orchestra.run.smithi031.stdout:1922 read 0x24f55b9 thru 0x24f64f0 (0xf38 bytes) 2024-09-13T07:12:51.113 INFO:teuthology.orchestra.run.smithi031.stdout:1924 read 0x3eba520 thru 0x3ebadb9 (0x89a bytes) 2024-09-13T07:12:51.114 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:51.114+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:51.114 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:51.115 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:51.118 INFO:teuthology.orchestra.run.smithi031.stdout:1925 write 0x5fc4d79 thru 0x5fd2596 (0xd81e bytes) 2024-09-13T07:12:51.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:51.130 INFO:teuthology.orchestra.run.smithi031.stdout:1927 trunc from 0x5fd2597 to 0xe30dbc0 2024-09-13T07:12:51.135 INFO:teuthology.orchestra.run.smithi031.stdout:1928 punch from 0xcfb9f1f to 0xcfc9a1f, (0xfb00 bytes) 2024-09-13T07:12:51.137 INFO:teuthology.orchestra.run.smithi031.stdout:1930 trunc from 0xe30dbc0 to 0xcfc7147 2024-09-13T07:12:51.142 INFO:teuthology.orchestra.run.smithi031.stdout:1931 write 0x31daa40 thru 0x31db56b (0xb2c bytes) 2024-09-13T07:12:51.145 INFO:teuthology.orchestra.run.smithi031.stdout:1932 writesame 0x5c43b48 thru 0x5c455e4 data_size 0x3(0x1a9d bytes) 2024-09-13T07:12:51.148 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.148 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.149 INFO:teuthology.orchestra.run.smithi031.stdout:1935 punch from 0x2ddb1cf to 0x2de08bf, (0x56f0 bytes) 2024-09-13T07:12:51.149 INFO:teuthology.orchestra.run.smithi031.stdout:1936 punch from 0xa3aaa96 to 0xa3b3420, (0x898a bytes) 2024-09-13T07:12:51.149 INFO:teuthology.orchestra.run.smithi031.stdout:1937 write 0x40a8dcd thru 0x40b542b (0xc65f bytes) 2024-09-13T07:12:51.151 INFO:teuthology.orchestra.run.smithi031.stdout:1938 read 0x4802ad2 thru 0x480aa49 (0x7f78 bytes) 2024-09-13T07:12:51.151 INFO:teuthology.orchestra.run.smithi031.stdout:1940 read 0x6d05863 thru 0x6d1021d (0xa9bb bytes) 2024-09-13T07:12:51.151 INFO:teuthology.orchestra.run.smithi031.stdout:1941 write 0xb46f304 thru 0xb479d8d (0xaa8a bytes) 2024-09-13T07:12:51.162 INFO:teuthology.orchestra.run.smithi031.stdout:1942 trunc from 0xcfc7147 to 0x56043f8 2024-09-13T07:12:51.185 INFO:teuthology.orchestra.run.smithi031.stdout:1944 write 0xaf9b750 thru 0xaf9e724 (0x2fd5 bytes) 2024-09-13T07:12:51.187 INFO:teuthology.orchestra.run.smithi031.stdout:1945 write 0xa1295c5 thru 0xa12f1b9 (0x5bf5 bytes) 2024-09-13T07:12:51.190 INFO:teuthology.orchestra.run.smithi031.stdout:1947 write 0x1f2755c thru 0x1f33bc4 (0xc669 bytes) 2024-09-13T07:12:51.193 INFO:teuthology.orchestra.run.smithi031.stdout:1949 punch from 0x2afdf67 to 0x2b06024, (0x80bd bytes) 2024-09-13T07:12:51.193 INFO:teuthology.orchestra.run.smithi031.stdout:1950 read 0x1117eee thru 0x1122b91 (0xaca4 bytes) 2024-09-13T07:12:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:1951 trunc from 0xaf9e725 to 0x7233f97 2024-09-13T07:12:51.208 INFO:teuthology.orchestra.run.smithi031.stdout:1952 read 0x55df1b6 thru 0x55e37ce (0x4619 bytes) 2024-09-13T07:12:51.208 INFO:teuthology.orchestra.run.smithi031.stdout:1953 read 0x296c451 thru 0x296c77b (0x32b bytes) 2024-09-13T07:12:51.208 INFO:teuthology.orchestra.run.smithi031.stdout:1955 write 0x170daa0 thru 0x171d023 (0xf584 bytes) 2024-09-13T07:12:51.211 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.211 INFO:teuthology.orchestra.run.smithi031.stdout:1957 read 0x5d5a26f thru 0x5d62813 (0x85a5 bytes) 2024-09-13T07:12:51.215 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:51.215+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:51.219 INFO:teuthology.orchestra.run.smithi031.stdout:1958 write 0x942cce8 thru 0x943774b (0xaa64 bytes) 2024-09-13T07:12:51.230 INFO:teuthology.orchestra.run.smithi031.stdout:1960 writesame 0xc09e14b thru 0xc0ad494 data_size 0x26(0xf34a bytes) 2024-09-13T07:12:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:1961 trunc from 0xc0ad495 to 0x4e1bd3d 2024-09-13T07:12:51.248 INFO:teuthology.orchestra.run.smithi031.stdout:1962 writesame 0x3ae049f thru 0x3aec4b8 data_size 0x43(0xc01a bytes) 2024-09-13T07:12:51.250 INFO:teuthology.orchestra.run.smithi031.stdout:1963 write 0x2437a54 thru 0x2439d14 (0x22c1 bytes) 2024-09-13T07:12:51.253 INFO:teuthology.orchestra.run.smithi031.stdout:1964 read 0x41067b9 thru 0x410ba94 (0x52dc bytes) 2024-09-13T07:12:51.253 INFO:teuthology.orchestra.run.smithi031.stdout:1965 read 0x261054e thru 0x261fdde (0xf891 bytes) 2024-09-13T07:12:51.253 INFO:teuthology.orchestra.run.smithi031.stdout:1966 flatten 2024-09-13T07:12:51.255 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:51.255 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:51.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:51.255 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4257.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.670761991999999, 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01398422, 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4257", 2024-09-13T07:12:51.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/806497074", 2024-09-13T07:12:51.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T07:12:51.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:51.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:12:51.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584054+0000", 2024-09-13T07:12:51.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584061+0000", 2024-09-13T07:12:51.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1929999999999998e-06 2024-09-13T07:12:51.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584069+0000", 2024-09-13T07:12:51.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8240000000000006e-06 2024-09-13T07:12:51.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584077+0000", 2024-09-13T07:12:51.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T07:12:51.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584133+0000", 2024-09-13T07:12:51.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5896000000000002e-05 2024-09-13T07:12:51.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:12:51.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584145+0000", 2024-09-13T07:12:51.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2293000000000001e-05 2024-09-13T07:12:51.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593792+0000", 2024-09-13T07:12:51.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096463009999999995 2024-09-13T07:12:51.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593914+0000", 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121994 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.594023+0000", 2024-09-13T07:12:51.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109152 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597204+0000", 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003181021 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597968+0000", 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076444399999999997 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597998+0000", 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0051000000000001e-05 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.598040+0000", 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1548999999999998e-05 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7104 3.19 3:9ddc4a4c:::rbd_data.11736f46b2f8.0000000000000007:9 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693202+0000", 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.5616159810000001, 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085172330000000008, 2024-09-13T07:12:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7104 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693202+0000", 2024-09-13T07:12:51.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693202+0000", 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693201+0000", 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693204+0000", 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.543e-06 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693214+0000", 2024-09-13T07:12:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0336999999999999e-05 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693220+0000", 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701613+0000", 2024-09-13T07:12:51.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083930240000000007 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701672+0000", 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9108999999999999e-05 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701719+0000", 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7120000000000003e-05 2024-09-13T07:12:51.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7112 3.19 3:98be421e:::rbd_data.11736f46b2f8.0000000000000006:9 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693284+0000", 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.5615334330000001, 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008649126, 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:51.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7112 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693284+0000", 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693284+0000", 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693284+0000", 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693297+0000", 2024-09-13T07:12:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2927e-05 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693306+0000", 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9970000000000004e-06 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693311+0000", 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-13T07:12:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701759+0000", 2024-09-13T07:12:51.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084480839999999998 2024-09-13T07:12:51.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701842+0000", 2024-09-13T07:12:51.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2599e-05 2024-09-13T07:12:51.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701933+0000", 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1346000000000003e-05 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7113 3.19 3:98be421e:::rbd_data.11736f46b2f8.0000000000000006:9 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.5615012849999999, 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088087640000000002, 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7113 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693318+0000", 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6759999999999999e-06 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693324+0000", 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8900000000000004e-06 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693329+0000", 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9139999999999999e-06 2024-09-13T07:12:51.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701974+0000", 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086443170000000003 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702050+0000", 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6386999999999999e-05 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702125+0000", 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5057999999999999e-05 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7114 3.19 3:9ddc4a4c:::rbd_data.11736f46b2f8.0000000000000007:9 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:51.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.561483585, 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089814739999999997, 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7114 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:12:51.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693336+0000", 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2340000000000001e-06 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693344+0000", 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3310000000000004e-06 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693349+0000", 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1549999999999998e-06 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702155+0000", 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088061089999999995 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702237+0000", 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1180999999999994e-05 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702315+0000", 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8845999999999997e-05 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7115 3.19 3:9ddc4a4c:::rbd_data.11736f46b2f8.0000000000000007:9 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.5614631650000002, 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091461110000000002, 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7115 2024-09-13T07:12:51.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:12:51.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693357+0000", 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693365+0000", 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2300000000000008e-06 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693370+0000", 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702342+0000", 2024-09-13T07:12:51.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089718839999999994 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702420+0000", 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7986999999999997e-05 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702500+0000", 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0829999999999997e-05 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7130 3.19 3:9ddc4a4c:::rbd_data.11736f46b2f8.0000000000000007:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.695192+0000", 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.5596253089999998, 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084284800000000003, 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7130 2024-09-13T07:12:51.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.695192+0000", 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.695192+0000", 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.695192+0000", 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.695203+0000", 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1586000000000001e-05 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.695211+0000", 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.763e-06 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.695217+0000", 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.703465+0000", 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082481069999999993 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.703545+0000", 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0246000000000004e-05 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.703621+0000", 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5263000000000001e-05 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7725 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8259832060000001, 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086548819999999992, 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7725 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428838+0000", 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3359999999999997e-06 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428847+0000", 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5289999999999997e-06 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428852+0000", 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0130000000000001e-06 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.432776+0000", 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039239380000000001 2024-09-13T07:12:51.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.432867+0000", 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0299999999999999e-05 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433090+0000", 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022336900000000001 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434316+0000", 2024-09-13T07:12:51.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001225547 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437408+0000", 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030926180000000001 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437455+0000", 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437489+0000", 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4663000000000003e-05 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7726 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.428964+0000", 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8258534119999998, 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093293319999999992, 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7726 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428964+0000", 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428964+0000", 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428963+0000", 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428969+0000", 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428980+0000", 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1435e-05 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428987+0000", 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5710000000000003e-06 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433485+0000", 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044981860000000004 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433558+0000", 2024-09-13T07:12:51.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3308999999999999e-05 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433958+0000", 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000399923 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438195+0000", 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042367280000000004 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438204+0000", 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6349999999999995e-06 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438252+0000", 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8922999999999999e-05 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438293+0000", 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0989999999999999e-05 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7727 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429188+0000", 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.825629467, 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092464539999999994, 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7727 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429188+0000", 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429188+0000", 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429187+0000", 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429207+0000", 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.995e-05 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429227+0000", 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0097000000000001e-05 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429236+0000", 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0920000000000004e-06 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434341+0000", 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051047599999999999 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434417+0000", 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5958999999999996e-05 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434619+0000", 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020198699999999999 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438306+0000", 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036867649999999998 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438352+0000", 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879000000000003e-05 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438397+0000", 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5068000000000003e-05 2024-09-13T07:12:51.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438434+0000", 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7045000000000002e-05 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7728 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429262+0000", 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.825555504, 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093129170000000004, 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7728 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429262+0000", 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429262+0000", 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429261+0000", 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429268+0000", 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.392e-06 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429277+0000", 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8049999999999996e-06 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429282+0000", 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434983+0000", 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057009419999999996 2024-09-13T07:12:51.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435037+0000", 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3696000000000002e-05 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435238+0000", 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000200674 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438311+0000", 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030733829999999998 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438495+0000", 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184079 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438540+0000", 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4428000000000002e-05 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438575+0000", 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5339000000000002e-05 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7729 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429393+0000", 2024-09-13T07:12:51.327 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8254247540000001, 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011020118000000001, 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7729 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429393+0000", 2024-09-13T07:12:51.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429393+0000", 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429392+0000", 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429396+0000", 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3080000000000001e-06 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429403+0000", 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5570000000000001e-06 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429407+0000", 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9720000000000003e-06 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435587+0000", 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061800049999999997 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435678+0000", 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1380999999999998e-05 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436091+0000", 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041248700000000001 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439596+0000", 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035056060000000001 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440367+0000", 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077025499999999996 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440393+0000", 2024-09-13T07:12:51.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6670999999999999e-05 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440413+0000", 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9538000000000002e-05 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7730 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429524+0000", 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8252936470000001, 2024-09-13T07:12:51.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010984203, 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7730 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429524+0000", 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429524+0000", 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429523+0000", 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429528+0000", 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.04e-06 2024-09-13T07:12:51.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429541+0000", 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867000000000001e-05 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429548+0000", 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0299999999999996e-06 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436671+0000", 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071234640000000004 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436756+0000", 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4546e-05 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437081+0000", 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032500999999999998 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439914+0000", 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028333020000000002 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440450+0000", 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053587299999999997 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440475+0000", 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4763e-05 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440508+0000", 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3229999999999999e-05 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7731 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429641+0000", 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.825176532, 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010952033, 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7731 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429641+0000", 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429641+0000", 2024-09-13T07:12:51.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429640+0000", 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429646+0000", 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4339999999999997e-06 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429660+0000", 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429668+0000", 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7520000000000002e-06 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437534+0000", 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078658840000000001 2024-09-13T07:12:51.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437586+0000", 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1758999999999997e-05 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437783+0000", 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019671700000000001 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439920+0000", 2024-09-13T07:12:51.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021370719999999998 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440549+0000", 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00062917400000000003 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440575+0000", 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6261000000000001e-05 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440593+0000", 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7657000000000001e-05 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7966 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665139+0000", 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.589678613, 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086921749999999999, 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7966 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665139+0000", 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665139+0000", 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665138+0000", 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665144+0000", 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0140000000000004e-06 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665156+0000", 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1579999999999999e-05 2024-09-13T07:12:51.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665166+0000", 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0081999999999999e-05 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.670352+0000", 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051865929999999998 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.670426+0000", 2024-09-13T07:12:51.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3530999999999998e-05 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.670928+0000", 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050263100000000002 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672306+0000", 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001378017 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673782+0000", 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001475924 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673812+0000", 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9261999999999999e-05 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673831+0000", 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9500999999999998e-05 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7967 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665216+0000", 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.589601378, 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087581970000000006, 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7967 2024-09-13T07:12:51.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665216+0000", 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665216+0000", 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665215+0000", 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665220+0000", 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9830000000000004e-06 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665230+0000", 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7890000000000006e-06 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665236+0000", 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.671394+0000", 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061572270000000004 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.671458+0000", 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4614000000000002e-05 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.671747+0000", 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028857699999999999 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673842+0000", 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020953999999999999 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673888+0000", 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5401000000000001e-05 2024-09-13T07:12:51.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673932+0000", 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4373999999999999e-05 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673974+0000", 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2412e-05 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7968 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665245+0000", 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5895725860000001, 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096363979999999991, 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7968 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665245+0000", 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665245+0000", 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665244+0000", 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665248+0000", 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6859999999999999e-06 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665256+0000", 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7470000000000005e-06 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665261+0000", 2024-09-13T07:12:51.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-06 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672332+0000", 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070711209999999997 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672422+0000", 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9598999999999994e-05 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672724+0000", 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030285300000000001 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674028+0000", 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013034279999999999 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674800+0000", 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077240800000000004 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674847+0000", 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6371e-05 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674881+0000", 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4656000000000002e-05 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7969 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665344+0000", 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5894732760000001, 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098855350000000008, 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7969 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665344+0000", 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665344+0000", 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665343+0000", 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665349+0000", 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6799999999999998e-06 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665359+0000", 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665366+0000", 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673121+0000", 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077545390000000004 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673203+0000", 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2095000000000006e-05 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673512+0000", 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030892000000000002 2024-09-13T07:12:51.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674951+0000", 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00143925 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675166+0000", 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000214976 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675194+0000", 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8390000000000001e-05 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675230+0000", 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5318999999999998e-05 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7970 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.368 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665529+0000", 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5892887309999999, 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098965349999999997, 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7970 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665529+0000", 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665529+0000", 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665528+0000", 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665533+0000", 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9359999999999998e-06 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665547+0000", 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4453e-05 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665555+0000", 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8099999999999998e-06 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674060+0000", 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085048309999999992 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674136+0000", 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5694999999999999e-05 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674440+0000", 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030381700000000001 2024-09-13T07:12:51.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675235+0000", 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079571899999999996 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675339+0000", 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010414800000000001 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675389+0000", 2024-09-13T07:12:51.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9529e-05 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675425+0000", 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6396999999999997e-05 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:8347 3.19 3:9f4f66e3:::rbd_data.117344605995.0000000000000168:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.070765+0000", 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.1840529719999999, 2024-09-13T07:12:51.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013952958999999999, 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 8347 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070765+0000", 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070765+0000", 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070764+0000", 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070769+0000", 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8720000000000001e-06 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070778+0000", 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1900000000000001e-06 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070783+0000", 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070820+0000", 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7472000000000003e-05 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070910+0000", 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9815000000000007e-05 2024-09-13T07:12:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.073366+0000", 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002455718 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084601+0000", 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011234675 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084647+0000", 2024-09-13T07:12:51.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084678+0000", 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1315000000000001e-05 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084717+0000", 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9643e-05 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.381 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:51.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:1967 punch from 0x1f2bfd3 to 0x1f2eb92, (0x2bbf bytes) 2024-09-13T07:12:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:1970 read 0x2acac92 thru 0x2acc4d1 (0x1840 bytes) 2024-09-13T07:12:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:1972 writesame 0x567d5e8 thru 0x5681626 data_size 0x1(0x403f bytes) 2024-09-13T07:12:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:1973 punch from 0x4daa826 to 0x4db15a1, (0x6d7b bytes) 2024-09-13T07:12:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:1977 clone 10 order 25 su 1048576 sc 11 2024-09-13T07:12:51.384 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:51.383+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:51.415 INFO:teuthology.orchestra.run.smithi184.stdout:get_pg_num_history pg_num_history pg_num_history(e30 pg_nums {1={14=8},2={15=1},3={19=32}} deleted_pools ) 2024-09-13T07:12:51.415 INFO:teuthology.orchestra.run.smithi184.stdout:Importing pgid 3.16 2024-09-13T07:12:51.415 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6847a25e:::rbd_data.11734b163e95.0000000000000013:head# 2024-09-13T07:12:51.415 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 0=[]:{} 2024-09-13T07:12:51.416 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:69bcfbd0:::rbd_data.1173f77ecad8.0000000000000063:head# 2024-09-13T07:12:51.416 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 6=[]:{} 2024-09-13T07:12:51.416 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:69d6b5f1:::rbd_data.1173f77ecad8.000000000000005b:head# 2024-09-13T07:12:51.417 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 6=[]:{} 2024-09-13T07:12:51.417 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6a67f01d:::rbd_data.1173f77ecad8.0000000000000027:head# 2024-09-13T07:12:51.417 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 6=[]:{} 2024-09-13T07:12:51.417 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6c86404b:::rbd_data.1173c53667c.0000000000000006:head# 2024-09-13T07:12:51.417 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 0=[]:{} 2024-09-13T07:12:51.418 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6c95c7f1:::rbd_data.11736f46b2f8.000000000000016a:head# 2024-09-13T07:12:51.418 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 0=[]:{} 2024-09-13T07:12:51.418 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6d931999:::rbd_data.1173f77ecad8.000000000000001c:head# 2024-09-13T07:12:51.418 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 6=[]:{} 2024-09-13T07:12:51.419 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6db07b17:::rbd_data.1173f77ecad8.0000000000000023:head# 2024-09-13T07:12:51.419 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 6=[]:{} 2024-09-13T07:12:51.419 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6dcc6547:::rbd_data.1173f77ecad8.0000000000000035:head# 2024-09-13T07:12:51.419 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 6=[]:{} 2024-09-13T07:12:51.420 INFO:teuthology.orchestra.run.smithi184.stdout:Write #3:6f3c8750:::rbd_data.1173f77ecad8.000000000000002f:head# 2024-09-13T07:12:51.420 INFO:teuthology.orchestra.run.smithi184.stdout:snapset 6=[]:{} 2024-09-13T07:12:51.420 INFO:teuthology.orchestra.run.smithi184.stdout:write_pg epoch 30 info 3.16( v 30'52 (0'0,30'52] local-lis/les=19/20 n=10 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-13T07:12:51.424 INFO:teuthology.orchestra.run.smithi184.stdout:Import successful 2024-09-13T07:12:51.483 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:51.484+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:51.529 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:51.529 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:51.529 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:51.529 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:51.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:51.584 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:51.585+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:51.644 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:51.645 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:51.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:51.698 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:51.699+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:51.760 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.462872375, 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:12:51.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:12:51.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.461394350000001, 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.458577301, 2024-09-13T07:12:51.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1319 3.0 e25/19 3:06065967:::rbd_data.1173c53667c.0000000000000056:head v 25'7, mlcod=25'5)", 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.287891906, 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026907649999999999, 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472901+0000", 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472909+0000", 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472925+0000", 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6019e-05 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472933+0000", 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8089999999999993e-06 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472979+0000", 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6034e-05 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.473001+0000", 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2218999999999999e-05 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475552+0000", 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025503309999999999 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475593+0000", 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1468999999999999e-05 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.889148341, 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:12:51.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.876524744999999, 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:51.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:12:51.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3179 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'130, mlcod=27'127)", 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.733242114999999, 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027322449999999999, 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027551+0000", 2024-09-13T07:12:51.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027558+0000", 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027569+0000", 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.059e-05 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027575+0000", 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5420000000000002e-06 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027617+0000", 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1328999999999997e-05 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027637+0000", 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0407999999999998e-05 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030260+0000", 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026228509999999998 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030284+0000", 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4389999999999999e-05 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1329429770000008, 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:12:51.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1324057500000002, 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:12:51.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.993324114, 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:12:51.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:12:51.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4162 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'113, mlcod=29'110)", 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9465163959999998, 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028573499999999998, 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814277+0000", 2024-09-13T07:12:51.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814283+0000", 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814293+0000", 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0299000000000001e-05 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814298+0000", 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:12:51.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814345+0000", 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7707999999999998e-05 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814365+0000", 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9836000000000001e-05 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817100+0000", 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027346969999999999 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817135+0000", 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5194999999999998e-05 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9459768210000004, 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:12:51.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:12:51.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9184927159999994, 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:12:51.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:12:51.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:12:51.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4195 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'122, mlcod=29'119)", 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9129571090000006, 2024-09-13T07:12:51.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027697870000000001, 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847836+0000", 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847841+0000", 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847850+0000", 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2390000000000002e-06 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847854+0000", 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847880+0000", 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:12:51.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847900+0000", 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0015000000000001e-05 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850568+0000", 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002668283 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850607+0000", 2024-09-13T07:12:51.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8349999999999997e-05 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9124782919999994, 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:12:51.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:12:51.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.8607859980000008, 2024-09-13T07:12:51.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:51.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:12:51.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:12:51.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3245621500000002, 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7730 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'61, mlcod=34'54)", 2024-09-13T07:12:51.833 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3236603709999999, 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028499979999999999, 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437133+0000", 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437139+0000", 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7409999999999998e-06 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437151+0000", 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1946e-05 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437157+0000", 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437201+0000", 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3917e-05 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437221+0000", 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439958+0000", 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027367300000000002 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439984+0000", 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5551999999999999e-05 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8060 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'136, mlcod=34'133)", 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.997307588, 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002785984, 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763486+0000", 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763492+0000", 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5050000000000004e-06 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763510+0000", 2024-09-13T07:12:51.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7833e-05 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763517+0000", 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763556+0000", 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8702999999999999e-05 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763577+0000", 2024-09-13T07:12:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1375000000000001e-05 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766246+0000", 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026686890000000001 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766273+0000", 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6930000000000001e-05 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5644089719999998, 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:51.846 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:51.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:51.847 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:51.848+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:51.934 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /var/log/ceph/exp.3.16.7 2024-09-13T07:12:51.947 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:51.948+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:51.994 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:51.994 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:52.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:52.049 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:52.050+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:52.150 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:52.150+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:52.150 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:52.150 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:52.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:52.251 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:52.251+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:52.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1098 3.15 e25/19 3:afeba560:::rbd_data.1173f77ecad8.000000000000004f:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.015190990000001, 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091051389999999999, 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.315 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6639999999999998e-06 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299484+0000", 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4609e-05 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299491+0000", 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.584e-06 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299534+0000", 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640999999999998e-05 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299553+0000", 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8564999999999999e-05 2024-09-13T07:12:52.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308532+0000", 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089789510000000006 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308568+0000", 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5896000000000003e-05 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.c epoch 25/19 rep_tid 48 entries 25'5 (0'0) error 3:32647bc6:::rbd_data.1173f77ecad8.0000000000000057:head by client.4467.0:1100 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'2)", 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.013393004999999, 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073257039999999997, 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:52.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301266+0000", 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5239999999999997e-06 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301279+0000", 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2856e-05 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301296+0000", 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6858e-05 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301369+0000", 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2631000000000003e-05 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308586+0000", 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072174989999999996 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1104 3.1e 3:7e73e8c7:::rbd_data.1173f77ecad8.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.012791545000001, 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009729985, 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1104 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301861+0000", 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301866+0000", 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301879+0000", 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3083e-05 2024-09-13T07:12:52.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301893+0000", 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4656e-05 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301950+0000", 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6671e-05 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302027+0000", 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6859000000000004e-05 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302260+0000", 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023318300000000001 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303132+0000", 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000871487 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311492+0000", 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083604839999999996 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311550+0000", 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7701000000000002e-05 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311592+0000", 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2024000000000003e-05 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1106 3.c e25/19 3:30d6dbcc:::rbd_data.1173f77ecad8.0000000000000050:head v 25'6, mlcod=25'3)", 2024-09-13T07:12:52.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.012211748, 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090851709999999995, 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302441+0000", 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302447+0000", 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7969999999999997e-06 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302460+0000", 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2772000000000001e-05 2024-09-13T07:12:52.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302466+0000", 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302507+0000", 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1014999999999999e-05 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302520+0000", 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2706e-05 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311485+0000", 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089648929999999998 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311527+0000", 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2259000000000003e-05 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.332 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1828 3.18 3:1c28841c:::rbd_data.1173c53667c.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write 963576~17461] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.168255744, 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085613430000000008, 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1828 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:52.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146396+0000", 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146417+0000", 2024-09-13T07:12:52.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1087999999999999e-05 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146431+0000", 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3788000000000001e-05 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146442+0000", 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0545e-05 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146535+0000", 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3067999999999997e-05 2024-09-13T07:12:52.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146542+0000", 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5290000000000003e-06 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148549+0000", 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002006407 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148675+0000", 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126549 2024-09-13T07:12:52.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148818+0000", 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014280900000000001 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.152082+0000", 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032638709999999998 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154899+0000", 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028167520000000001 2024-09-13T07:12:52.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154935+0000", 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6392e-05 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154959+0000", 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4170000000000001e-05 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3311 3.18 3:1ef9c6ed:::rbd_data.11734c6039c3.00000000000000c8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:52.338 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.145533797000001, 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02606054, 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3311 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169119+0000", 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169124+0000", 2024-09-13T07:12:52.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169134+0000", 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0325e-05 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169139+0000", 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.916e-06 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169183+0000", 2024-09-13T07:12:52.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4369999999999997e-05 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169319+0000", 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013582199999999999 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.173537+0000", 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042175870000000001 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195024+0000", 2024-09-13T07:12:52.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021487336999999999 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195119+0000", 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4963000000000005e-05 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195157+0000", 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7743000000000001e-05 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195180+0000", 2024-09-13T07:12:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3272e-05 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:3311 3.18 e27/19 ondisk, result = 0)", 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.128959346, 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093332329999999998, 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185693+0000", 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185700+0000", 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999996e-06 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185720+0000", 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0055e-05 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185730+0000", 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9709999999999999e-06 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195010+0000", 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092796179999999999 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195018+0000", 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1999999999999994e-06 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195027+0000", 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3335 3.b 3:d2887617:::rbd_data.11734c6039c3.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.059502999999999, 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018824244, 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3335 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255155+0000", 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9250000000000003e-06 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255165+0000", 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0682e-05 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255172+0000", 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1600000000000001e-06 2024-09-13T07:12:52.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255212+0000", 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9329999999999998e-05 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255260+0000", 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7981e-05 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.258002+0000", 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027423500000000002 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273857+0000", 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015854844999999999 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273892+0000", 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4938999999999999e-05 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273927+0000", 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5515999999999997e-05 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273975+0000", 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7307e-05 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4246 3.1e 3:78aceff8:::rbd_data.11736f46b2f8.00000000000000c8:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4142318669999998, 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017416903000000001, 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4246 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900421+0000", 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900426+0000", 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900437+0000", 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1062e-05 2024-09-13T07:12:52.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900442+0000", 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900480+0000", 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8012000000000001e-05 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900590+0000", 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109262 2024-09-13T07:12:52.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902585+0000", 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019956090000000002 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917754+0000", 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015168683000000001 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917788+0000", 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4190000000000003e-05 2024-09-13T07:12:52.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917817+0000", 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8486000000000001e-05 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917838+0000", 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1911000000000001e-05 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4247 3.7 3:e146183c:::rbd_data.11736f46b2f8.00000000000000c9:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4142024919999994, 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070352820000000003, 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4247 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900451+0000", 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900453+0000", 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:12:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900457+0000", 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9369999999999997e-06 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900460+0000", 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.937e-06 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900512+0000", 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2119e-05 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900622+0000", 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011024800000000001 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902136+0000", 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001513723 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907373+0000", 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052366859999999999 2024-09-13T07:12:52.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907409+0000", 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.693e-05 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907453+0000", 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3262e-05 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907486+0000", 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3541999999999998e-05 2024-09-13T07:12:52.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4249 3.1e 3:7927132b:::rbd_data.11736f46b2f8.00000000000000cf:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4100915569999994, 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013320103, 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4249 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.364 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904560+0000", 2024-09-13T07:12:52.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904568+0000", 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9489999999999996e-06 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904582+0000", 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3619e-05 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904589+0000", 2024-09-13T07:12:52.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4660000000000002e-06 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.910641+0000", 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060512719999999999 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.910732+0000", 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1306999999999998e-05 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.911772+0000", 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001039653 2024-09-13T07:12:52.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917783+0000", 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060117579999999999 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917844+0000", 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0378999999999999e-05 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917866+0000", 2024-09-13T07:12:52.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1858999999999999e-05 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917882+0000", 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6317999999999999e-05 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:4246 3.1e e29/19 ondisk, result = 0)", 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4067203359999993, 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098236480000000008, 2024-09-13T07:12:52.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907932+0000", 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907936+0000", 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9820000000000002e-06 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907945+0000", 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8179999999999993e-06 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907950+0000", 2024-09-13T07:12:52.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6500000000000004e-06 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917744+0000", 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097943720000000008 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917749+0000", 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8099999999999997e-06 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917757+0000", 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:6699 3.2 3:42a1fedc:::rbd_object_map.1173b1ea0ed4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.568273+0000", 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7463800389999999, 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070972479999999996, 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6699 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568273+0000", 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568273+0000", 2024-09-13T07:12:52.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568272+0000", 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568278+0000", 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6949999999999997e-06 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568289+0000", 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0499999999999999e-05 2024-09-13T07:12:52.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568295+0000", 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5300000000000002e-06 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.572074+0000", 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037785129999999998 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.572151+0000", 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7688000000000006e-05 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.572498+0000", 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034641099999999997 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.574640+0000", 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021418259999999999 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.575315+0000", 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067500899999999996 2024-09-13T07:12:52.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.575343+0000", 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8192e-05 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.575371+0000", 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7929999999999999e-05 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:6700 3.2 3:42a1fedc:::rbd_object_map.1173b1ea0ed4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.568464+0000", 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7461892219999999, 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069874760000000003, 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6700 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568464+0000", 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568464+0000", 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568463+0000", 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568470+0000", 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568479+0000", 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4790000000000002e-06 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.568487+0000", 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3499999999999999e-06 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.573039+0000", 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045523899999999999 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.573088+0000", 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8813999999999997e-05 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.573329+0000", 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024063900000000001 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.574678+0000", 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00134899 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.575381+0000", 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070317700000000003 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.575417+0000", 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5814e-05 2024-09-13T07:12:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.575452+0000", 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5097e-05 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7256 3.2 3:40781c20:::rbd_data.1173b1ea0ed4.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.834745+0000", 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.4799087640000002, 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086331080000000004, 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7256 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834745+0000", 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834745+0000", 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834744+0000", 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834749+0000", 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834758+0000", 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6690000000000001e-06 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834763+0000", 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7509999999999999e-06 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834792+0000", 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9048e-05 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834873+0000", 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1170999999999999e-05 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.836602+0000", 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017287330000000001 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843239+0000", 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066368909999999998 2024-09-13T07:12:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843284+0000", 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5466000000000001e-05 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843335+0000", 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0343000000000002e-05 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843378+0000", 2024-09-13T07:12:52.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3235000000000002e-05 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:8106 3.1e 3:7a058a18:::rbd_data.117344605995.0000000000000069:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.814368+0000", 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.500285163, 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094120939999999993, 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 8106 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814368+0000", 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814368+0000", 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814367+0000", 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814373+0000", 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814383+0000", 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0298e-05 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814388+0000", 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814432+0000", 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3611e-05 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814513+0000", 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1048000000000001e-05 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.816184+0000", 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016717489999999999 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823721+0000", 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075362099999999998 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823730+0000", 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4739999999999994e-06 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823760+0000", 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9465999999999999e-05 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823780+0000", 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0798999999999999e-05 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:8275 3.7 3:e054be62:::rbd_data.117344605995.000000000000009f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:52.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.965717+0000", 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3489364780000002, 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021118511999999999, 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 8275 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965717+0000", 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965717+0000", 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965715+0000", 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965724+0000", 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9840000000000007e-06 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965739+0000", 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4625e-05 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965746+0000", 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6640000000000001e-06 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965800+0000", 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4824999999999998e-05 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965961+0000", 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016047699999999999 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.986689+0000", 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020727668000000001 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.986748+0000", 2024-09-13T07:12:52.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9157000000000003e-05 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.986835+0000", 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7701999999999995e-05 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:9180 3.1e 3:7f7aa705:::rbd_data.11737e4ac805.00000000000000be:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.882415+0000", 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.432238731, 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021333556, 2024-09-13T07:12:52.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9180 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882415+0000", 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882415+0000", 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882413+0000", 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882420+0000", 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882432+0000", 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1776999999999999e-05 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882438+0000", 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882482+0000", 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3473000000000002e-05 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882581+0000", 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9388000000000005e-05 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.887222+0000", 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046404510000000003 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903580+0000", 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016358579000000002 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903630+0000", 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0111999999999997e-05 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903693+0000", 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3063000000000003e-05 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903748+0000", 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5013000000000003e-05 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:9180 3.1e e35/19 ondisk, result = 0)", 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.894780+0000", 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.419873366, 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089788569999999998, 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894780+0000", 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894780+0000", 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894779+0000", 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894786+0000", 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2559999999999996e-06 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894803+0000", 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7181e-05 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894815+0000", 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1729000000000001e-05 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903609+0000", 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087941870000000002 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903622+0000", 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3095000000000001e-05 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903759+0000", 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000136852 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:9182 3.8 3:150c1be9:::rbd_data.11737e4ac805.0000000000000026:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T07:12:52.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.905558+0000", 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4090951110000001, 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01705462, 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9182 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905558+0000", 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905558+0000", 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905557+0000", 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905563+0000", 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7549999999999997e-06 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905575+0000", 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1358e-05 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905581+0000", 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9680000000000003e-06 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905619+0000", 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8606999999999998e-05 2024-09-13T07:12:52.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905719+0000", 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9795000000000003e-05 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.908292+0000", 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025727010000000002 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922501+0000", 2024-09-13T07:12:52.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014208903 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922515+0000", 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4217e-05 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922569+0000", 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4209000000000002e-05 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:52.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922613+0000", 2024-09-13T07:12:52.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3862000000000001e-05 2024-09-13T07:12:52.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:52.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:52.415 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:52.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:12:52.416 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:52.417+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:52.417 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:52.418 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T07:12:52.420 INFO:teuthology.orchestra.run.smithi031.stdout:1978 punch from 0x13e01 to 0x154c3, (0x16c2 bytes) 2024-09-13T07:12:52.424 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:52.431 INFO:teuthology.orchestra.run.smithi031.stdout:1980 trunc from 0x5681627 to 0xbb4e3cf 2024-09-13T07:12:52.437 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:52.437 INFO:teuthology.orchestra.run.smithi031.stdout:1982 write 0xb9de4a6 thru 0xb9ea93e (0xc499 bytes) 2024-09-13T07:12:52.441 INFO:teuthology.orchestra.run.smithi031.stdout:1983 read 0x2e1e67d thru 0x2e1fb5d (0x14e1 bytes) 2024-09-13T07:12:52.441 INFO:teuthology.orchestra.run.smithi031.stdout:1984 write 0x57adccd thru 0x57b6d1d (0x9051 bytes) 2024-09-13T07:12:52.446 INFO:teuthology.orchestra.run.smithi031.stdout:1985 write 0x34df388 thru 0x34e3253 (0x3ecc bytes) 2024-09-13T07:12:52.451 INFO:teuthology.orchestra.run.smithi031.stdout:1986 write 0xacdbaf1 thru 0xace696f (0xae7f bytes) 2024-09-13T07:12:52.454 INFO:teuthology.orchestra.run.smithi031.stdout:1987 write 0xa7a5c1d thru 0xa7aa8a5 (0x4c89 bytes) 2024-09-13T07:12:52.458 INFO:teuthology.orchestra.run.smithi031.stdout:1988 punch from 0x39c8bd to 0x3a5f29, (0x966c bytes) 2024-09-13T07:12:52.471 INFO:teuthology.orchestra.run.smithi031.stdout:1989 trunc from 0xbb4e3cf to 0x1cdc7c5 2024-09-13T07:12:52.508 INFO:teuthology.orchestra.run.smithi031.stdout:1990 writesame 0x997c2c6 thru 0x99823ca data_size 0x11(0x6105 bytes) 2024-09-13T07:12:52.514 INFO:teuthology.orchestra.run.smithi031.stdout:1991 trunc from 0x99823cb to 0x5acb0d0 2024-09-13T07:12:52.517 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:52.518+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:52.524 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:52.524 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:52.524 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:52.524 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:52.533 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:52.533 INFO:teuthology.orchestra.run.smithi031.stdout:1992 write 0xb615bb7 thru 0xb61cf80 (0x73ca bytes) 2024-09-13T07:12:52.534 INFO:teuthology.orchestra.run.smithi031.stdout:1993 read 0xab018fd thru 0xab0db54 (0xc258 bytes) 2024-09-13T07:12:52.535 INFO:teuthology.orchestra.run.smithi031.stdout:1994 write 0x948cd7 thru 0x94c971 (0x3c9b bytes) 2024-09-13T07:12:52.536 INFO:teuthology.orchestra.run.smithi031.stdout:1995 read 0x73079ba thru 0x730d7bf (0x5e06 bytes) 2024-09-13T07:12:52.536 INFO:teuthology.orchestra.run.smithi031.stdout:1996 write 0x4d036b4 thru 0x4d06167 (0x2ab4 bytes) 2024-09-13T07:12:52.538 INFO:teuthology.orchestra.run.smithi031.stdout:1997 read 0x358e947 thru 0x3595cb9 (0x7373 bytes) 2024-09-13T07:12:52.538 INFO:teuthology.orchestra.run.smithi031.stdout:1998 punch from 0x1480712 to 0x1489c13, (0x9501 bytes) 2024-09-13T07:12:52.538 INFO:teuthology.orchestra.run.smithi031.stdout:1999 write 0x2d2e047 thru 0x2d38afe (0xaab8 bytes) 2024-09-13T07:12:52.540 INFO:teuthology.orchestra.run.smithi031.stdout:2000 punch from 0x82821be to 0x8284fef, (0x2e31 bytes) 2024-09-13T07:12:52.540 INFO:teuthology.orchestra.run.smithi031.stdout:2001 read 0x83d60a3 thru 0x83dcc2e (0x6b8c bytes) 2024-09-13T07:12:52.540 INFO:teuthology.orchestra.run.smithi031.stdout:2002 read 0x4477dac thru 0x447a794 (0x29e9 bytes) 2024-09-13T07:12:52.540 INFO:teuthology.orchestra.run.smithi031.stdout:2004 read 0x55e0d55 thru 0x55e7dce (0x707a bytes) 2024-09-13T07:12:52.549 INFO:teuthology.orchestra.run.smithi031.stdout:2005 write 0xe122ddc thru 0xe127896 (0x4abb bytes) 2024-09-13T07:12:52.550 INFO:teuthology.orchestra.run.smithi031.stdout:2006 read 0x295bc1 thru 0x2a4898 (0xecd8 bytes) 2024-09-13T07:12:52.551 INFO:teuthology.orchestra.run.smithi031.stdout:2007 writesame 0x37071e5 thru 0x37152f9 data_size 0x1(0xe115 bytes) 2024-09-13T07:12:52.581 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:52.581 INFO:teuthology.orchestra.run.smithi031.stdout:2009 write 0xd1b8458 thru 0xd1c19b9 (0x9562 bytes) 2024-09-13T07:12:52.582 INFO:teuthology.orchestra.run.smithi031.stdout:2010 read 0x34eea5c thru 0x34f7c26 (0x91cb bytes) 2024-09-13T07:12:52.582 INFO:teuthology.orchestra.run.smithi031.stdout:2011 read 0x1550ab7 thru 0x155dd66 (0xd2b0 bytes) 2024-09-13T07:12:52.583 INFO:teuthology.orchestra.run.smithi031.stdout:2012 read 0x9350f8e thru 0x935422d (0x32a0 bytes) 2024-09-13T07:12:52.583 INFO:teuthology.orchestra.run.smithi031.stdout:2013 write 0xaa80ad4 thru 0xaa8199e (0xecb bytes) 2024-09-13T07:12:52.584 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:52.584 INFO:teuthology.orchestra.run.smithi031.stdout:2014 write 0xab1b66d thru 0xab22923 (0x72b7 bytes) 2024-09-13T07:12:52.586 INFO:teuthology.orchestra.run.smithi031.stdout:2016 clone 11 order 23 su 65536 sc 12 2024-09-13T07:12:52.601 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:52.618 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:52.619+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:52.619 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:52.619 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:52.635 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:52.636 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:52.636 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:52.636 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:52.637 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:52.637 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:52.645 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:52.651 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:52.651 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:52.667 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:52.667 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:52.683 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:52.684 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:52.684 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:12:52.687 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:12:52.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:12:52.719 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:52.720+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:52.747 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:52.747 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:52.756 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:52.820 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:52.821+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:52.898 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:52.898 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:52.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:52.952 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:52.953+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:53.009 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.009 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.017 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:53.022 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:53.023+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:53.122 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:53.123+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:53.160 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.160 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.169 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:53.223 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:53.224+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:53.253 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:12:53.315 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone10 from 0xe127897 (overlap 0x1cdc7c5) to 0xe6e3e2 2024-09-13T07:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.317 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.323 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:53.324+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:53.327 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T07:12:53.425 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:53.425+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:53.431 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.431 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.440 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:53.478 INFO:teuthology.orchestra.run.smithi031.stdout:2017 punch from 0x80bed37 to 0x80ce06f, (0xf338 bytes) 2024-09-13T07:12:53.482 INFO:teuthology.orchestra.run.smithi031.stdout:2018 punch from 0x8c4f459 to 0x8c5196d, (0x2514 bytes) 2024-09-13T07:12:53.482 INFO:teuthology.orchestra.run.smithi031.stdout:2019 read 0x536fc61 thru 0x537e7c5 (0xeb65 bytes) 2024-09-13T07:12:53.482 INFO:teuthology.orchestra.run.smithi031.stdout:2020 punch from 0x26f272b to 0x26fba24, (0x92f9 bytes) 2024-09-13T07:12:53.483 INFO:teuthology.orchestra.run.smithi031.stdout:2022 punch from 0xa685d91 to 0xa692f6b, (0xd1da bytes) 2024-09-13T07:12:53.483 INFO:teuthology.orchestra.run.smithi031.stdout:2023 write 0x110ef8a thru 0x111c888 (0xd8ff bytes) 2024-09-13T07:12:53.496 INFO:teuthology.orchestra.run.smithi031.stdout:2024 trunc from 0xe127897 to 0xd5a29fb 2024-09-13T07:12:53.526 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:53.527+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:53.534 INFO:teuthology.orchestra.run.smithi031.stdout:2025 read 0x896a40b thru 0x896a942 (0x538 bytes) 2024-09-13T07:12:53.534 INFO:teuthology.orchestra.run.smithi031.stdout:2028 writesame 0xaf0e20a thru 0xaf13779 data_size 0x4(0x5570 bytes) 2024-09-13T07:12:53.548 INFO:teuthology.orchestra.run.smithi031.stdout:2029 write 0xc887fc7 thru 0xc88b65b (0x3695 bytes) 2024-09-13T07:12:53.550 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.550 INFO:teuthology.orchestra.run.smithi031.stdout:2031 writesame 0xb1ff117 thru 0xb20b1b1 data_size 0x1(0xc09b bytes) 2024-09-13T07:12:53.561 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.561 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:53.575 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.575 INFO:teuthology.orchestra.run.smithi031.stdout:2033 read 0x9e816b6 thru 0x9e8d39e (0xbce9 bytes) 2024-09-13T07:12:53.589 INFO:teuthology.orchestra.run.smithi031.stdout:2035 trunc from 0xd5a29fb to 0x1414434 2024-09-13T07:12:53.626 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:53.627+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:53.664 INFO:teuthology.orchestra.run.smithi031.stdout:2036 trunc from 0x1414434 to 0x54abfc7 2024-09-13T07:12:53.673 INFO:teuthology.orchestra.run.smithi031.stdout:2037 writesame 0x58f3386 thru 0x58f9a99 data_size 0x12(0x6714 bytes) 2024-09-13T07:12:53.674 INFO:teuthology.orchestra.run.smithi031.stdout:2039 read 0x24b06c1 thru 0x24bd5aa (0xceea bytes) 2024-09-13T07:12:53.675 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.675 INFO:teuthology.orchestra.run.smithi031.stdout:2041 read 0x39811c2 thru 0x39846be (0x34fd bytes) 2024-09-13T07:12:53.685 INFO:teuthology.orchestra.run.smithi031.stdout:2042 writesame 0x9a9c092 thru 0x9aa3c03 data_size 0x17(0x7b72 bytes) 2024-09-13T07:12:53.685 INFO:teuthology.orchestra.run.smithi031.stdout:2043 write 0x59b085f thru 0x59b32c1 (0x2a63 bytes) 2024-09-13T07:12:53.686 INFO:teuthology.orchestra.run.smithi031.stdout:2045 punch from 0x30e94b5 to 0x30f8b91, (0xf6dc bytes) 2024-09-13T07:12:53.686 INFO:teuthology.orchestra.run.smithi031.stdout:2046 write 0x17b3e90 thru 0x17c0651 (0xc7c2 bytes) 2024-09-13T07:12:53.688 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.688 INFO:teuthology.orchestra.run.smithi031.stdout:2048 writesame 0x48c742 thru 0x49127f data_size 0x1(0x4b3e bytes) 2024-09-13T07:12:53.689 INFO:teuthology.orchestra.run.smithi031.stdout:2050 read 0x759de35 thru 0x75aca23 (0xebef bytes) 2024-09-13T07:12:53.689 INFO:teuthology.orchestra.run.smithi031.stdout:2051 read 0x271b8e5 thru 0x271b940 (0x5c bytes) 2024-09-13T07:12:53.690 INFO:teuthology.orchestra.run.smithi031.stdout:2052 write 0x822596c thru 0x8233547 (0xdbdc bytes) 2024-09-13T07:12:53.693 INFO:teuthology.orchestra.run.smithi031.stdout:2053 punch from 0x58eefd9 to 0x58f0727, (0x174e bytes) 2024-09-13T07:12:53.695 INFO:teuthology.orchestra.run.smithi031.stdout:2054 trunc from 0x9aa3c04 to 0x7df051a 2024-09-13T07:12:53.702 INFO:teuthology.orchestra.run.smithi031.stdout:2055 writesame 0x72357fa thru 0x7241c77 data_size 0x7(0xc47e bytes) 2024-09-13T07:12:53.710 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.711 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.715 INFO:teuthology.orchestra.run.smithi031.stdout:2056 trunc from 0x7df051a to 0xbc6e0a3 2024-09-13T07:12:53.720 INFO:teuthology.orchestra.run.smithi031.stdout:2057 read 0x681e117 thru 0x6826f3f (0x8e29 bytes) 2024-09-13T07:12:53.720 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.720 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:53.732 INFO:teuthology.orchestra.run.smithi031.stdout:2062 trunc from 0xbc6e0a3 to 0x2242603 2024-09-13T07:12:53.732 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:53.732+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:53.763 INFO:teuthology.orchestra.run.smithi031.stdout:2064 write 0xc68f29a thru 0xc6986a9 (0x9410 bytes) 2024-09-13T07:12:53.770 INFO:teuthology.orchestra.run.smithi031.stdout:2065 read 0xa703de3 thru 0xa70627f (0x249d bytes) 2024-09-13T07:12:53.781 INFO:teuthology.orchestra.run.smithi031.stdout:2067 trunc from 0xc6986aa to 0x291ae02 2024-09-13T07:12:53.815 INFO:teuthology.orchestra.run.smithi031.stdout:2068 write 0xcb5e781 thru 0xcb6ca8b (0xe30b bytes) 2024-09-13T07:12:53.820 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.825 INFO:teuthology.orchestra.run.smithi031.stdout:2071 writesame 0xce62c80 thru 0xce6e996 data_size 0x1(0xbd17 bytes) 2024-09-13T07:12:53.826 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.826 INFO:teuthology.orchestra.run.smithi031.stdout:2073 read 0x5aa2f11 thru 0x5aab7a8 (0x8898 bytes) 2024-09-13T07:12:53.827 INFO:teuthology.orchestra.run.smithi031.stdout:2075 read 0x4d40cf7 thru 0x4d46f38 (0x6242 bytes) 2024-09-13T07:12:53.827 INFO:teuthology.orchestra.run.smithi031.stdout:2076 read 0x91997da thru 0x919b504 (0x1d2b bytes) 2024-09-13T07:12:53.828 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:53.828+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:53.828 INFO:teuthology.orchestra.run.smithi031.stdout:2078 trunc from 0xce6e997 to 0xbae1089 2024-09-13T07:12:53.844 INFO:teuthology.orchestra.run.smithi031.stdout:2079 write 0xd11a6f6 thru 0xd11f028 (0x4933 bytes) 2024-09-13T07:12:53.852 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.852 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.855 INFO:teuthology.orchestra.run.smithi031.stdout:2082 trunc from 0xd11f029 to 0x5632bbb 2024-09-13T07:12:53.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:53.873 INFO:teuthology.orchestra.run.smithi031.stdout:2083 trunc from 0x5632bbb to 0x69063c7 2024-09-13T07:12:53.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.881 INFO:teuthology.orchestra.run.smithi031.stdout:2085 trunc from 0x69063c7 to 0x4ef9cee 2024-09-13T07:12:53.893 INFO:teuthology.orchestra.run.smithi031.stdout:2087 read 0x27719a5 thru 0x277e390 (0xc9ec bytes) 2024-09-13T07:12:53.893 INFO:teuthology.orchestra.run.smithi031.stdout:2088 read 0x2bc6444 thru 0x2bd4abe (0xe67b bytes) 2024-09-13T07:12:53.893 INFO:teuthology.orchestra.run.smithi031.stdout:2090 read 0x2f82c68 thru 0x2f8cd89 (0xa122 bytes) 2024-09-13T07:12:53.894 INFO:teuthology.orchestra.run.smithi031.stdout:2092 punch from 0xa7f852 to 0xa8c28b, (0xca39 bytes) 2024-09-13T07:12:53.894 INFO:teuthology.orchestra.run.smithi031.stdout:2093 read 0x4dd041a thru 0x4dda9c4 (0xa5ab bytes) 2024-09-13T07:12:53.894 INFO:teuthology.orchestra.run.smithi031.stdout:2095 read 0x313c98a thru 0x313e7bd (0x1e34 bytes) 2024-09-13T07:12:53.896 INFO:teuthology.orchestra.run.smithi031.stdout:2098 trunc from 0x4ef9cee to 0x40c8af0 2024-09-13T07:12:53.907 INFO:teuthology.orchestra.run.smithi031.stdout:2099 punch from 0x1c296d0 to 0x1c2b600, (0x1f30 bytes) 2024-09-13T07:12:53.907 INFO:teuthology.orchestra.run.smithi031.stdout:2100 write 0x1774cf5 thru 0x177d88a (0x8b96 bytes) 2024-09-13T07:12:53.909 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.924 INFO:teuthology.orchestra.run.smithi031.stdout:2103 write 0xd4f169d thru 0xd4f720d (0x5b71 bytes) 2024-09-13T07:12:53.927 INFO:teuthology.orchestra.run.smithi031.stdout:2105 read 0x5e76a8b thru 0x5e777bf (0xd35 bytes) 2024-09-13T07:12:53.927 INFO:teuthology.orchestra.run.smithi031.stdout:2109 read 0xa64342c thru 0xa65152e (0xe103 bytes) 2024-09-13T07:12:53.927 INFO:teuthology.orchestra.run.smithi031.stdout:2110 read 0x98de013 thru 0x98eaf68 (0xcf56 bytes) 2024-09-13T07:12:53.927 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.928 INFO:teuthology.orchestra.run.smithi031.stdout:2115 read 0xac7e79c thru 0xac898aa (0xb10f bytes) 2024-09-13T07:12:53.928 INFO:teuthology.orchestra.run.smithi031.stdout:2118 write 0xb74448e thru 0xb745263 (0xdd6 bytes) 2024-09-13T07:12:53.929 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:53.928+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:53.930 INFO:teuthology.orchestra.run.smithi031.stdout:2119 write 0xa633f89 thru 0xa642413 (0xe48b bytes) 2024-09-13T07:12:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:2122 write 0xb5e8d48 thru 0xb5f4403 (0xb6bc bytes) 2024-09-13T07:12:53.937 INFO:teuthology.orchestra.run.smithi031.stdout:2123 clone 12 order 21 su 8192 sc 13 2024-09-13T07:12:53.987 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:53.987 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:53.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:54.030 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:54.031+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:54.131 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:54.130+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:54.141 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:54.141 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:54.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:54.234 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:54.234+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:54.301 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:54.301 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:54.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:54.321 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone11 from 0xd4f720e (overlap 0x1414434) to 0xe0e2f1 2024-09-13T07:12:54.365 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:54.365+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:54.407 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T07:12:54.427 INFO:teuthology.orchestra.run.smithi031.stdout:2125 read 0xaf51227 thru 0xaf5feae (0xec88 bytes) 2024-09-13T07:12:54.428 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.428 INFO:teuthology.orchestra.run.smithi031.stdout:2127 writesame 0x1090a9d thru 0x109f3ca data_size 0x3(0xe92e bytes) 2024-09-13T07:12:54.434 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:54.434+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:54.440 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:54.440 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:54.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:54.533 INFO:teuthology.orchestra.run.smithi031.stdout:2128 write 0x1877303 thru 0x1886107 (0xee05 bytes) 2024-09-13T07:12:54.534 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:54.535+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:54.576 INFO:teuthology.orchestra.run.smithi031.stdout:2131 punch from 0x12b8d4c to 0x12ba895, (0x1b49 bytes) 2024-09-13T07:12:54.576 INFO:teuthology.orchestra.run.smithi031.stdout:2132 write 0xbee2a7c thru 0xbee6dbb (0x4340 bytes) 2024-09-13T07:12:54.596 INFO:teuthology.orchestra.run.smithi031.stdout:2133 writesame 0x3106eee thru 0x31167ec data_size 0x1(0xf8ff bytes) 2024-09-13T07:12:54.635 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:54.635+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:54.653 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:54.653 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:54.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:54.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:12:54.726 INFO:teuthology.orchestra.run.smithi031.stdout:2134 read 0xa2b8f8 thru 0xa322d9 (0x69e2 bytes) 2024-09-13T07:12:54.727 INFO:teuthology.orchestra.run.smithi031.stdout:2135 punch from 0x98e55b4 to 0x98ec185, (0x6bd1 bytes) 2024-09-13T07:12:54.727 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.727 INFO:teuthology.orchestra.run.smithi031.stdout:2137 writesame 0x5e9d4a9 thru 0x5e9ffe8 data_size 0x20(0x2b40 bytes) 2024-09-13T07:12:54.736 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:54.737+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:54.745 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.746 INFO:teuthology.orchestra.run.smithi031.stdout:2141 read 0x516ea11 thru 0x5173f04 (0x54f4 bytes) 2024-09-13T07:12:54.746 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.746 INFO:teuthology.orchestra.run.smithi031.stdout:2144 writesame 0x1b2e9ce thru 0x1b34012 data_size 0x7(0x5645 bytes) 2024-09-13T07:12:54.764 INFO:teuthology.orchestra.run.smithi031.stdout:2145 read 0x201d295 thru 0x202ad52 (0xdabe bytes) 2024-09-13T07:12:54.772 INFO:teuthology.orchestra.run.smithi031.stdout:2146 trunc from 0xd4f720e to 0x88b4b88 2024-09-13T07:12:54.837 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:54.838+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:54.851 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:54.851 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:54.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:54.865 INFO:teuthology.orchestra.run.smithi031.stdout:2147 read 0x5b11d4 thru 0x5bec17 (0xda44 bytes) 2024-09-13T07:12:54.867 INFO:teuthology.orchestra.run.smithi031.stdout:2149 punch from 0x319c89 to 0x325cec, (0xc063 bytes) 2024-09-13T07:12:54.873 INFO:teuthology.orchestra.run.smithi031.stdout:2150 trunc from 0x88b4b88 to 0x1fe5d70 2024-09-13T07:12:54.876 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:54.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:54.909 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:54.927 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:54.932 INFO:teuthology.orchestra.run.smithi031.stdout:2153 write 0x621984f thru 0x621a55c (0xd0e bytes) 2024-09-13T07:12:54.938 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:54.938+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:54.942 INFO:teuthology.orchestra.run.smithi031.stdout:2155 write 0x7b06e71 thru 0x7b0c3dd (0x556d bytes) 2024-09-13T07:12:54.946 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:54.947 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.959 INFO:teuthology.orchestra.run.smithi031.stdout:2158 writesame 0xbfe2cbd thru 0xbfea66f data_size 0x43(0x79b3 bytes) 2024-09-13T07:12:54.959 INFO:teuthology.orchestra.run.smithi031.stdout:2161 read 0x1d56083 thru 0x1d56592 (0x510 bytes) 2024-09-13T07:12:54.960 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.960 INFO:teuthology.orchestra.run.smithi031.stdout:2163 read 0x5fcb6c9 thru 0x5fd6877 (0xb1af bytes) 2024-09-13T07:12:54.960 INFO:teuthology.orchestra.run.smithi031.stdout:2164 write 0xba88cae thru 0xba963c9 (0xd71c bytes) 2024-09-13T07:12:54.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:54.970 INFO:teuthology.orchestra.run.smithi031.stdout:2165 write 0x263c4ac thru 0x2648954 (0xc4a9 bytes) 2024-09-13T07:12:54.972 INFO:teuthology.orchestra.run.smithi031.stdout:2166 read 0x8dfde3c thru 0x8e0288d (0x4a52 bytes) 2024-09-13T07:12:54.972 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.972 INFO:teuthology.orchestra.run.smithi031.stdout:2169 read 0xb4e11 thru 0xbf56a (0xa75a bytes) 2024-09-13T07:12:54.973 INFO:teuthology.orchestra.run.smithi031.stdout:2170 write 0x9ddef00 thru 0x9dea6f1 (0xb7f2 bytes) 2024-09-13T07:12:54.984 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:2171 read 0x7b33865 thru 0x7b36ae1 (0x327d bytes) 2024-09-13T07:12:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:2173 punch from 0x52f39c7 to 0x52fca31, (0x906a bytes) 2024-09-13T07:12:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:2174 read 0x626bb58 thru 0x6275ff5 (0xa49e bytes) 2024-09-13T07:12:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:2176 read 0x7485729 thru 0x748579c (0x74 bytes) 2024-09-13T07:12:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:2177 writesame 0x4d612eb thru 0x4d6918c data_size 0x9(0x7ea2 bytes) 2024-09-13T07:12:54.994 INFO:teuthology.orchestra.run.smithi031.stdout:2178 write 0x44bd72a thru 0x44bfa62 (0x2339 bytes) 2024-09-13T07:12:55.004 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:55.005 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:55.005 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:12:55.005 INFO:teuthology.orchestra.run.smithi031.stdout:2179 write 0xdaefd03 thru 0xdafaa9a (0xad98 bytes) 2024-09-13T07:12:55.008 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.711019563000001, 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:12:55.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.709541538, 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:55.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:12:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.706724488999999, 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1319 3.0 e25/19 3:06065967:::rbd_data.1173c53667c.0000000000000056:head v 25'7, mlcod=25'5)", 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.536039093999999, 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026907649999999999, 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472902+0000", 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472901+0000", 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472909+0000", 2024-09-13T07:12:55.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472925+0000", 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6019e-05 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472933+0000", 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8089999999999993e-06 2024-09-13T07:12:55.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.472979+0000", 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6034e-05 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.473001+0000", 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2218999999999999e-05 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475552+0000", 2024-09-13T07:12:55.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025503309999999999 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.475593+0000", 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1468999999999999e-05 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.137295528999999, 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.124671933, 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:12:55.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:12:55.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3179 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'130, mlcod=27'127)", 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.981389303, 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027322449999999999, 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027551+0000", 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027558+0000", 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027569+0000", 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.059e-05 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027575+0000", 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5420000000000002e-06 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027617+0000", 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1328999999999997e-05 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027637+0000", 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0407999999999998e-05 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030260+0000", 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026228509999999998 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030284+0000", 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4389999999999999e-05 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.381090165, 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:12:55.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:12:55.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:12:55.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:12:55.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:12:55.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.380552937999999, 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:12:55.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:12:55.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.241471302000001, 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:12:55.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:12:55.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4162 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'113, mlcod=29'110)", 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.194663584000001, 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028573499999999998, 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814277+0000", 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814283+0000", 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814293+0000", 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0299000000000001e-05 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814298+0000", 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:12:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814345+0000", 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7707999999999998e-05 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814365+0000", 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9836000000000001e-05 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817100+0000", 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027346969999999999 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817135+0000", 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5194999999999998e-05 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:55.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.194124008999999, 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:12:55.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.166639904, 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:12:55.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:12:55.067 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:12:55.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:12:55.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4195 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'122, mlcod=29'119)", 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.161104297, 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027697870000000001, 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847836+0000", 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847841+0000", 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847850+0000", 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2390000000000002e-06 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847854+0000", 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847880+0000", 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847900+0000", 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0015000000000001e-05 2024-09-13T07:12:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850568+0000", 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002668283 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850607+0000", 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8349999999999997e-05 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.16062548, 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:12:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:12:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:12:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.108933186, 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:12:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5727093380000001, 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:12:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7730 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'61, mlcod=34'54)", 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5718075589999998, 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028499979999999999, 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437133+0000", 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437139+0000", 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7409999999999998e-06 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437151+0000", 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1946e-05 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437157+0000", 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437201+0000", 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3917e-05 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437221+0000", 2024-09-13T07:12:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439958+0000", 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027367300000000002 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439984+0000", 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5551999999999999e-05 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8060 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'136, mlcod=34'133)", 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2454547759999999, 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002785984, 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:12:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763486+0000", 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763492+0000", 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5050000000000004e-06 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763510+0000", 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7833e-05 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763517+0000", 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763556+0000", 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8702999999999999e-05 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763577+0000", 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1375000000000001e-05 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766246+0000", 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026686890000000001 2024-09-13T07:12:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766273+0000", 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6930000000000001e-05 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8125561599999997, 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:12:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:12:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.102 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:55.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:55.104 INFO:teuthology.orchestra.run.smithi031.stdout:2180 flatten 2024-09-13T07:12:55.104 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.104 INFO:teuthology.orchestra.run.smithi031.stdout:2183 trunc from 0xdafaa9b to 0xc44c40d 2024-09-13T07:12:55.104 INFO:teuthology.orchestra.run.smithi031.stdout:2184 read 0x34fb4fa thru 0x3500911 (0x5418 bytes) 2024-09-13T07:12:55.104 INFO:teuthology.orchestra.run.smithi031.stdout:2186 write 0x4bd7802 thru 0x4bdfce4 (0x84e3 bytes) 2024-09-13T07:12:55.104 INFO:teuthology.orchestra.run.smithi031.stdout:2187 write 0x31dfe5 thru 0x326173 (0x818f bytes) 2024-09-13T07:12:55.104 INFO:teuthology.orchestra.run.smithi031.stdout:2188 read 0x535d85e thru 0x536c443 (0xebe6 bytes) 2024-09-13T07:12:55.105 INFO:teuthology.orchestra.run.smithi031.stdout:2189 write 0xd4f3d65 thru 0xd4f4d83 (0x101f bytes) 2024-09-13T07:12:55.105 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.105 INFO:teuthology.orchestra.run.smithi031.stdout:2191 punch from 0x912e30f to 0x91374c8, (0x91b9 bytes) 2024-09-13T07:12:55.105 INFO:teuthology.orchestra.run.smithi031.stdout:2192 write 0xc4d93b8 thru 0xc4e6023 (0xcc6c bytes) 2024-09-13T07:12:55.105 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:55.104+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:55.107 INFO:teuthology.orchestra.run.smithi031.stdout:2194 trunc from 0xd4f4d84 to 0x4356974 2024-09-13T07:12:55.125 INFO:teuthology.orchestra.run.smithi031.stdout:2196 writesame 0x7cff85 thru 0x7d50ee data_size 0x2(0x516a bytes) 2024-09-13T07:12:55.130 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.130 INFO:teuthology.orchestra.run.smithi031.stdout:2199 read 0xce4b2d thru 0xcea45e (0x5932 bytes) 2024-09-13T07:12:55.130 INFO:teuthology.orchestra.run.smithi031.stdout:2200 read 0x1e24279 thru 0x1e31218 (0xcfa0 bytes) 2024-09-13T07:12:55.131 INFO:teuthology.orchestra.run.smithi031.stdout:2201 read 0xf2e2df thru 0xf35e5a (0x7b7c bytes) 2024-09-13T07:12:55.132 INFO:teuthology.orchestra.run.smithi031.stdout:2203 read 0xc922dd thru 0xc94681 (0x23a5 bytes) 2024-09-13T07:12:55.133 INFO:teuthology.orchestra.run.smithi031.stdout:2205 read 0x3c86a4a thru 0x3c8ea72 (0x8029 bytes) 2024-09-13T07:12:55.145 INFO:teuthology.orchestra.run.smithi031.stdout:2206 writesame 0x92a0042 thru 0x92a9c17 data_size 0x3d(0x9bd6 bytes) 2024-09-13T07:12:55.145 INFO:teuthology.orchestra.run.smithi031.stdout:2207 read 0x6943c1d thru 0x694abce (0x6fb2 bytes) 2024-09-13T07:12:55.145 INFO:teuthology.orchestra.run.smithi031.stdout:2210 punch from 0x8c32a13 to 0x8c35d69, (0x3356 bytes) 2024-09-13T07:12:55.145 INFO:teuthology.orchestra.run.smithi031.stdout:2212 read 0x2fc928 thru 0x309378 (0xca51 bytes) 2024-09-13T07:12:55.155 INFO:teuthology.orchestra.run.smithi031.stdout:2213 writesame 0xc77b440 thru 0xc78a086 data_size 0x1(0xec47 bytes) 2024-09-13T07:12:55.155 INFO:teuthology.orchestra.run.smithi031.stdout:2214 write 0x2601671 thru 0x260b513 (0x9ea3 bytes) 2024-09-13T07:12:55.157 INFO:teuthology.orchestra.run.smithi031.stdout:2215 writesame 0xba337ba thru 0xba342dd data_size 0x17(0xb24 bytes) 2024-09-13T07:12:55.162 INFO:teuthology.orchestra.run.smithi031.stdout:2216 trunc from 0xc78a087 to 0xcc6e6df 2024-09-13T07:12:55.167 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.167 INFO:teuthology.orchestra.run.smithi031.stdout:2219 read 0x856bbf2 thru 0x857aa9d (0xeeac bytes) 2024-09-13T07:12:55.168 INFO:teuthology.orchestra.run.smithi031.stdout:2220 write 0x9199057 thru 0x91a0b89 (0x7b33 bytes) 2024-09-13T07:12:55.175 INFO:teuthology.orchestra.run.smithi031.stdout:2221 read 0x4b53dab thru 0x4b5aa11 (0x6c67 bytes) 2024-09-13T07:12:55.175 INFO:teuthology.orchestra.run.smithi031.stdout:2222 write 0xb54ebe1 thru 0xb5585f0 (0x9a10 bytes) 2024-09-13T07:12:55.200 INFO:teuthology.orchestra.run.smithi031.stdout:2223 trunc from 0xcc6e6df to 0xb7100a 2024-09-13T07:12:55.205 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:55.205+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:55.229 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:55.230 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:55.230 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:12:55.230 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:55.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:55.244 INFO:teuthology.orchestra.run.smithi031.stdout:2224 writesame 0x8aee6d0 thru 0x8aeec86 data_size 0x7(0x5b7 bytes) 2024-09-13T07:12:55.244 INFO:teuthology.orchestra.run.smithi031.stdout:2225 read 0x1e3136 thru 0x1ec357 (0x9222 bytes) 2024-09-13T07:12:55.245 INFO:teuthology.orchestra.run.smithi031.stdout:2226 punch from 0x790fa70 to 0x7913ee1, (0x4471 bytes) 2024-09-13T07:12:55.245 INFO:teuthology.orchestra.run.smithi031.stdout:2227 read 0x2a4fd1 thru 0x2b1f55 (0xcf85 bytes) 2024-09-13T07:12:55.251 INFO:teuthology.orchestra.run.smithi031.stdout:2232 trunc from 0x8aeec87 to 0xdb61a51 2024-09-13T07:12:55.257 INFO:teuthology.orchestra.run.smithi031.stdout:2234 read 0xb138a1e thru 0xb148128 (0xf70b bytes) 2024-09-13T07:12:55.270 INFO:teuthology.orchestra.run.smithi031.stdout:2235 trunc from 0xdb61a51 to 0x24ccdcb 2024-09-13T07:12:55.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:12:55.276 INFO:teuthology.orchestra.run.smithi031.stdout:2237 trunc from 0x24ccdcb to 0x3050ee6 2024-09-13T07:12:55.285 INFO:teuthology.orchestra.run.smithi031.stdout:2238 trunc from 0x3050ee6 to 0x2976b0 2024-09-13T07:12:55.295 INFO:teuthology.orchestra.run.smithi031.stdout:2239 read 0x18c2d9 thru 0x18f2e3 (0x300b bytes) 2024-09-13T07:12:55.296 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.296 INFO:teuthology.orchestra.run.smithi031.stdout:2244 punch from 0x17f427 to 0x185a04, (0x65dd bytes) 2024-09-13T07:12:55.296 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.297 INFO:teuthology.orchestra.run.smithi031.stdout:2247 trunc from 0x2976b0 to 0x1590a3f 2024-09-13T07:12:55.306 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:55.306+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:55.321 INFO:teuthology.orchestra.run.smithi031.stdout:2249 write 0xd114bcb thru 0xd1216e3 (0xcb19 bytes) 2024-09-13T07:12:55.330 INFO:teuthology.orchestra.run.smithi031.stdout:2251 punch from 0x2bcd1d4 to 0x2bd47be, (0x75ea bytes) 2024-09-13T07:12:55.333 INFO:teuthology.orchestra.run.smithi031.stdout:2253 trunc from 0xd1216e4 to 0xa6603fe 2024-09-13T07:12:55.343 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.344 INFO:teuthology.orchestra.run.smithi031.stdout:2255 punch from 0x6f5138c to 0x6f60824, (0xf498 bytes) 2024-09-13T07:12:55.349 INFO:teuthology.orchestra.run.smithi031.stdout:2257 trunc from 0xa6603fe to 0x54ee9c2 2024-09-13T07:12:55.361 INFO:teuthology.orchestra.run.smithi031.stdout:2258 trunc from 0x54ee9c2 to 0xa672301 2024-09-13T07:12:55.366 INFO:teuthology.orchestra.run.smithi031.stdout:2259 read 0x26a2873 thru 0x26a95fd (0x6d8b bytes) 2024-09-13T07:12:55.366 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.371 INFO:teuthology.orchestra.run.smithi031.stdout:2261 write 0xace8a84 thru 0xacebfb3 (0x3530 bytes) 2024-09-13T07:12:55.375 INFO:teuthology.orchestra.run.smithi031.stdout:2263 write 0x27ab9e8 thru 0x27b760e (0xbc27 bytes) 2024-09-13T07:12:55.377 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:55.377 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:12:55.377 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:12:55.377 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:12:55.377 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:55.383 INFO:teuthology.orchestra.run.smithi031.stdout:2266 read 0x1e95095 thru 0x1ea5084 (0xfff0 bytes) 2024-09-13T07:12:55.383 INFO:teuthology.orchestra.run.smithi031.stdout:2268 read 0x792cab2 thru 0x793b3da (0xe929 bytes) 2024-09-13T07:12:55.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:55.392 INFO:teuthology.orchestra.run.smithi031.stdout:2270 write 0xda3cd3d thru 0xda459a4 (0x8c68 bytes) 2024-09-13T07:12:55.402 INFO:teuthology.orchestra.run.smithi031.stdout:2273 read 0x7c817bc thru 0x7c8ed32 (0xd577 bytes) 2024-09-13T07:12:55.407 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:55.407+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:55.413 INFO:teuthology.orchestra.run.smithi031.stdout:2274 trunc from 0xda459a5 to 0x4a6fc27 2024-09-13T07:12:55.435 INFO:teuthology.orchestra.run.smithi031.stdout:2275 write 0x9bc3f5d thru 0x9bccf29 (0x8fcd bytes) 2024-09-13T07:12:55.443 INFO:teuthology.orchestra.run.smithi031.stdout:2276 writesame 0x96786c7 thru 0x96812bc data_size 0x5(0x8bf6 bytes) 2024-09-13T07:12:55.459 INFO:teuthology.orchestra.run.smithi031.stdout:2277 write 0xebab91d thru 0xebb159e (0x5c82 bytes) 2024-09-13T07:12:55.472 INFO:teuthology.orchestra.run.smithi031.stdout:2278 trunc from 0xebb159f to 0x9b4c36b 2024-09-13T07:12:55.483 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.490 INFO:teuthology.orchestra.run.smithi031.stdout:2283 trunc from 0x9b4c36b to 0x3649a2c 2024-09-13T07:12:55.508 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:55.508+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:55.510 INFO:teuthology.orchestra.run.smithi031.stdout:2284 writesame 0x78959d8 thru 0x78a57ae data_size 0x1(0xfdd7 bytes) 2024-09-13T07:12:55.517 INFO:teuthology.orchestra.run.smithi031.stdout:2285 trunc from 0x78a57af to 0x1d3ddfa 2024-09-13T07:12:55.522 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:12:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1098 3.15 e25/19 3:afeba560:::rbd_data.1173f77ecad8.000000000000004f:head v 25'8, mlcod=25'6)", 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.222933163, 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091051389999999999, 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299462+0000", 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:12:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6639999999999998e-06 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299484+0000", 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4609e-05 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299491+0000", 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.584e-06 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299534+0000", 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640999999999998e-05 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299553+0000", 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8564999999999999e-05 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308532+0000", 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089789510000000006 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308568+0000", 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5896000000000003e-05 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.c epoch 25/19 rep_tid 48 entries 25'5 (0'0) error 3:32647bc6:::rbd_data.1173f77ecad8.0000000000000057:head by client.4467.0:1100 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'2)", 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.221135178000001, 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073257039999999997, 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301260+0000", 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301266+0000", 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5239999999999997e-06 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301279+0000", 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2856e-05 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301296+0000", 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6858e-05 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301369+0000", 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2631000000000003e-05 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.308586+0000", 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072174989999999996 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1104 3.1e 3:7e73e8c7:::rbd_data.1173f77ecad8.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.220533717999999, 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009729985, 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1104 2024-09-13T07:12:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301862+0000", 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301861+0000", 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301866+0000", 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301879+0000", 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3083e-05 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301893+0000", 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4656e-05 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301950+0000", 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6671e-05 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302027+0000", 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6859000000000004e-05 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302260+0000", 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023318300000000001 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.303132+0000", 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000871487 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311492+0000", 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083604839999999996 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311550+0000", 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7701000000000002e-05 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311592+0000", 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2024000000000003e-05 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1106 3.c e25/19 3:30d6dbcc:::rbd_data.1173f77ecad8.0000000000000050:head v 25'6, mlcod=25'3)", 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.219953920999998, 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090851709999999995, 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302442+0000", 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302441+0000", 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302447+0000", 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7969999999999997e-06 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302460+0000", 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2772000000000001e-05 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302466+0000", 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302507+0000", 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1014999999999999e-05 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302520+0000", 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2706e-05 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311485+0000", 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089648929999999998 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.311527+0000", 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2259000000000003e-05 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:1828 3.18 3:1c28841c:::rbd_data.1173c53667c.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write 963576~17461] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.375997916999999, 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085613430000000008, 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1828 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146398+0000", 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146396+0000", 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146417+0000", 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1087999999999999e-05 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146431+0000", 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3788000000000001e-05 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146442+0000", 2024-09-13T07:12:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0545e-05 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146535+0000", 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3067999999999997e-05 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.146542+0000", 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5290000000000003e-06 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148549+0000", 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002006407 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148675+0000", 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126549 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.148818+0000", 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014280900000000001 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.152082+0000", 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032638709999999998 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154899+0000", 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028167520000000001 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154935+0000", 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6392e-05 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:38.154959+0000", 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4170000000000001e-05 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3311 3.18 3:1ef9c6ed:::rbd_data.11734c6039c3.00000000000000c8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.35327597, 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02606054, 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3311 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169120+0000", 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169119+0000", 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169124+0000", 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169134+0000", 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0325e-05 2024-09-13T07:12:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169139+0000", 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.916e-06 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169183+0000", 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4369999999999997e-05 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.169319+0000", 2024-09-13T07:12:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013582199999999999 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.173537+0000", 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042175870000000001 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195024+0000", 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021487336999999999 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195119+0000", 2024-09-13T07:12:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4963000000000005e-05 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195157+0000", 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7743000000000001e-05 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195180+0000", 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3272e-05 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:3311 3.18 e27/19 ondisk, result = 0)", 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.336701519, 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093332329999999998, 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185694+0000", 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185693+0000", 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185700+0000", 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999996e-06 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185720+0000", 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0055e-05 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.185730+0000", 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9709999999999999e-06 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195010+0000", 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092796179999999999 2024-09-13T07:12:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195018+0000", 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1999999999999994e-06 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.195027+0000", 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:3335 3.b 3:d2887617:::rbd_data.11734c6039c3.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.267245172999999, 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018824244, 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3335 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255150+0000", 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255155+0000", 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9250000000000003e-06 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255165+0000", 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0682e-05 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255172+0000", 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1600000000000001e-06 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255212+0000", 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9329999999999998e-05 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.255260+0000", 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7981e-05 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.258002+0000", 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027423500000000002 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273857+0000", 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015854844999999999 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273892+0000", 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4938999999999999e-05 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273927+0000", 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5515999999999997e-05 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.273975+0000", 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7307e-05 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4246 3.1e 3:78aceff8:::rbd_data.11736f46b2f8.00000000000000c8:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.62197404, 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017416903000000001, 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4246 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900422+0000", 2024-09-13T07:12:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900421+0000", 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900426+0000", 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900437+0000", 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1062e-05 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900442+0000", 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900480+0000", 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8012000000000001e-05 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900590+0000", 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109262 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902585+0000", 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019956090000000002 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917754+0000", 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015168683000000001 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917788+0000", 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4190000000000003e-05 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917817+0000", 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8486000000000001e-05 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917838+0000", 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1911000000000001e-05 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:4249 3.1e 3:7927132b:::rbd_data.11736f46b2f8.00000000000000cf:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.617833729999999, 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013320103, 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4249 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904562+0000", 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904560+0000", 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904568+0000", 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9489999999999996e-06 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904582+0000", 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3619e-05 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.904589+0000", 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4660000000000002e-06 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.910641+0000", 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060512719999999999 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.910732+0000", 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1306999999999998e-05 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.911772+0000", 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001039653 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917783+0000", 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060117579999999999 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917844+0000", 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0378999999999999e-05 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917866+0000", 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1858999999999999e-05 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917882+0000", 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6317999999999999e-05 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:4246 3.1e e29/19 ondisk, result = 0)", 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.614462508999999, 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098236480000000008, 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907933+0000", 2024-09-13T07:12:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907932+0000", 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907936+0000", 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9820000000000002e-06 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907945+0000", 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8179999999999993e-06 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.907950+0000", 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6500000000000004e-06 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917744+0000", 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097943720000000008 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917749+0000", 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8099999999999997e-06 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.917757+0000", 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7256 3.2 3:40781c20:::rbd_data.1173b1ea0ed4.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.834745+0000", 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.6876509369999999, 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086331080000000004, 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7256 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834745+0000", 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834745+0000", 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834744+0000", 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834749+0000", 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834758+0000", 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6690000000000001e-06 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834763+0000", 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7509999999999999e-06 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834792+0000", 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9048e-05 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.834873+0000", 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1170999999999999e-05 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.836602+0000", 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017287330000000001 2024-09-13T07:12:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843239+0000", 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066368909999999998 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843284+0000", 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5466000000000001e-05 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843335+0000", 2024-09-13T07:12:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0343000000000002e-05 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.843378+0000", 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3235000000000002e-05 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:8106 3.1e 3:7a058a18:::rbd_data.117344605995.0000000000000069:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.814368+0000", 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7080273359999998, 2024-09-13T07:12:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094120939999999993, 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 8106 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814368+0000", 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814368+0000", 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814367+0000", 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814373+0000", 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T07:12:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814383+0000", 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0298e-05 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814388+0000", 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814432+0000", 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3611e-05 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.814513+0000", 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1048000000000001e-05 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.816184+0000", 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016717489999999999 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823721+0000", 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075362099999999998 2024-09-13T07:12:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823730+0000", 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4739999999999994e-06 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823760+0000", 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9465999999999999e-05 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.823780+0000", 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0798999999999999e-05 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:8275 3.7 3:e054be62:::rbd_data.117344605995.000000000000009f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.965717+0000", 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5566786510000004, 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021118511999999999, 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 8275 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965717+0000", 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965717+0000", 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965715+0000", 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965724+0000", 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9840000000000007e-06 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965739+0000", 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4625e-05 2024-09-13T07:12:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965746+0000", 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6640000000000001e-06 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965800+0000", 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4824999999999998e-05 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.965961+0000", 2024-09-13T07:12:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016047699999999999 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.986689+0000", 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020727668000000001 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.986748+0000", 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9157000000000003e-05 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.986835+0000", 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7701999999999995e-05 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:9180 3.1e 3:7f7aa705:::rbd_data.11737e4ac805.00000000000000be:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.882415+0000", 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6399809039999997, 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021333556, 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9180 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882415+0000", 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882415+0000", 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882413+0000", 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882420+0000", 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882432+0000", 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1776999999999999e-05 2024-09-13T07:12:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882438+0000", 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882482+0000", 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3473000000000002e-05 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.882581+0000", 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9388000000000005e-05 2024-09-13T07:12:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.887222+0000", 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046404510000000003 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903580+0000", 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016358579000000002 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903630+0000", 2024-09-13T07:12:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0111999999999997e-05 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903693+0000", 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3063000000000003e-05 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903748+0000", 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5013000000000003e-05 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:9180 3.1e e35/19 ondisk, result = 0)", 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.894780+0000", 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6276155389999998, 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089788569999999998, 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894780+0000", 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894780+0000", 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894779+0000", 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894786+0000", 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2559999999999996e-06 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894803+0000", 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7181e-05 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.894815+0000", 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1729000000000001e-05 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903609+0000", 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087941870000000002 2024-09-13T07:12:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903622+0000", 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3095000000000001e-05 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.903759+0000", 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000136852 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:9182 3.8 3:150c1be9:::rbd_data.11737e4ac805.0000000000000026:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.905558+0000", 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6168372839999998, 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01705462, 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9182 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905558+0000", 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905558+0000", 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905557+0000", 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905563+0000", 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7549999999999997e-06 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905575+0000", 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1358e-05 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905581+0000", 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9680000000000003e-06 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905619+0000", 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8606999999999998e-05 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.905719+0000", 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9795000000000003e-05 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.908292+0000", 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025727010000000002 2024-09-13T07:12:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922501+0000", 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014208903 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922515+0000", 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4217e-05 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922569+0000", 2024-09-13T07:12:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4209000000000002e-05 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.922613+0000", 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3862000000000001e-05 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:9867 3.b 3:d13c33b0:::rbd_data.11734aa2fccf.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:52.553211+0000", 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.9691843470000001, 2024-09-13T07:12:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02919486, 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9867 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553211+0000", 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553211+0000", 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553210+0000", 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553216+0000", 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553225+0000", 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050000000000006e-06 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553229+0000", 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2390000000000004e-06 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553277+0000", 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7432999999999998e-05 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.553357+0000", 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0646e-05 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.559306+0000", 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059485220000000004 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.582336+0000", 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023029628999999999 2024-09-13T07:12:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.582343+0000", 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8590000000000005e-06 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.582379+0000", 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5772999999999998e-05 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.582406+0000", 2024-09-13T07:12:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-05 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:9867 3.b e36/19 ondisk, result = 0)", 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:52.572454+0000", 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.949941699, 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098838599999999995, 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.572454+0000", 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.572454+0000", 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.572453+0000", 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.572458+0000", 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0889999999999999e-06 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.572471+0000", 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2853e-05 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.572480+0000", 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4360000000000002e-06 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.582324+0000", 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098442819999999993 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.582331+0000", 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0840000000000003e-06 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:52.582338+0000", 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8070000000000002e-06 2024-09-13T07:12:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:22188 3.2 3:42be56b8:::rbd_data.1173f6773d36.000000000000000b:head [truncate 917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:55.219624+0000", 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.30277191399999998, 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076605550000000003, 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22188 2024-09-13T07:12:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.219624+0000", 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.219624+0000", 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.219623+0000", 2024-09-13T07:12:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.219626+0000", 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2969999999999999e-06 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.219636+0000", 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1640000000000007e-06 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.219642+0000", 2024-09-13T07:12:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6540000000000002e-06 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.221270+0000", 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016277450000000001 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.221357+0000", 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7169999999999994e-05 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T07:12:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.221486+0000", 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012833699999999999 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.222335+0000", 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084990200000000004 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.227237+0000", 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049012730000000003 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:12:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.227264+0000", 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7291999999999999e-05 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:55.227284+0000", 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0270000000000001e-05 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:12:55.624 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:55.625 INFO:teuthology.orchestra.run.smithi031.stdout:2286 read 0x13e5cf6 thru 0x13f03cf (0xa6da bytes) 2024-09-13T07:12:55.625 INFO:teuthology.orchestra.run.smithi031.stdout:2287 read 0x9c7993 thru 0x9d4253 (0xc8c1 bytes) 2024-09-13T07:12:55.626 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2288 writesame 0x3f9ddf5 thru 0x3fa3c5f data_size 0x7(0x5e6b bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2290 trunc from 0x3fa3c60 to 0x413a6d1 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2293 read 0x382722f thru 0x382c4f3 (0x52c5 bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2294 read 0x3d1b9e thru 0x3db8c9 (0x9d2c bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2296 write 0x3f15798 thru 0x3f22717 (0xcf80 bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2297 writesame 0x7606010 thru 0x760d92d data_size 0x25(0x791e bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2298 read 0x4329bf3 thru 0x432d17a (0x3588 bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2299 writesame 0xa6f0867 thru 0xa6fcf98 data_size 0x9(0xc732 bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2300 punch from 0x5223015 to 0x522d3ac, (0xa397 bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2303 punch from 0x8a3f3fc to 0x8a46f2b, (0x7b2f bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2304 punch from 0x9e9550f to 0x9e9c8c5, (0x73b6 bytes) 2024-09-13T07:12:55.629 INFO:teuthology.orchestra.run.smithi031.stdout:2305 read 0x7905a8c thru 0x7913750 (0xdcc5 bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2306 write 0xbbc0640 thru 0xbbcf6be (0xf07f bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2307 read 0x4f034ea thru 0x4f10d21 (0xd838 bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2308 trunc from 0xbbcf6bf to 0x9c5cd58 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2309 read 0x2c8c1f3 thru 0x2c9600d (0x9e1b bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2310 write 0x17f8bb7 thru 0x17fc243 (0x368d bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2311 read 0x5eadce9 thru 0x5eb9da7 (0xc0bf bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2313 read 0xc1a302 thru 0xc23799 (0x9498 bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2316 punch from 0x2b79cdf to 0x2b83ef0, (0xa211 bytes) 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2317 trunc from 0x9c5cd58 to 0xe4eeccf 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.630 INFO:teuthology.orchestra.run.smithi031.stdout:2320 trunc from 0xe4eeccf to 0xcb44394 2024-09-13T07:12:55.631 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:55.625+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:55.631 INFO:teuthology.orchestra.run.smithi031.stdout:2321 write 0xd2f6f36 thru 0xd305e31 (0xeefc bytes) 2024-09-13T07:12:55.648 INFO:teuthology.orchestra.run.smithi031.stdout:2322 trunc from 0xd305e32 to 0x2ec5a9d 2024-09-13T07:12:55.662 INFO:teuthology.orchestra.run.smithi031.stdout:2325 writesame 0x5847fa thru 0x58a4f2 data_size 0x1(0x5cf9 bytes) 2024-09-13T07:12:55.676 INFO:teuthology.orchestra.run.smithi031.stdout:2326 trunc from 0x2ec5a9d to 0x9da5dff 2024-09-13T07:12:55.682 INFO:teuthology.orchestra.run.smithi031.stdout:2327 punch from 0x72db5ab to 0x72defc8, (0x3a1d bytes) 2024-09-13T07:12:55.687 INFO:teuthology.orchestra.run.smithi031.stdout:2329 trunc from 0x9da5dff to 0xddc6161 2024-09-13T07:12:55.692 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.692 INFO:teuthology.orchestra.run.smithi031.stdout:2331 write 0x53c26e thru 0x54684d (0xa5e0 bytes) 2024-09-13T07:12:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:2332 read 0x8298061 thru 0x82a407a (0xc01a bytes) 2024-09-13T07:12:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:2334 write 0x161acd0 thru 0x1621d0f (0x7040 bytes) 2024-09-13T07:12:55.698 INFO:teuthology.orchestra.run.smithi031.stdout:2335 read 0x82a63db thru 0x82a92cb (0x2ef1 bytes) 2024-09-13T07:12:55.698 INFO:teuthology.orchestra.run.smithi031.stdout:2336 read 0x9a9d8d4 thru 0x9aa4bac (0x72d9 bytes) 2024-09-13T07:12:55.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.698 INFO:teuthology.orchestra.run.smithi031.stdout:2338 punch from 0xa34ccd1 to 0xa351ce7, (0x5016 bytes) 2024-09-13T07:12:55.703 INFO:teuthology.orchestra.run.smithi031.stdout:2339 trunc from 0xddc6161 to 0x8dd3a1d 2024-09-13T07:12:55.709 INFO:teuthology.orchestra.run.smithi031.stdout:2340 writesame 0x18be290 thru 0x18c370a data_size 0x3(0x547b bytes) 2024-09-13T07:12:55.712 INFO:teuthology.orchestra.run.smithi031.stdout:2341 read 0x63e686 thru 0x64a0af (0xba2a bytes) 2024-09-13T07:12:55.712 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:55.713 INFO:teuthology.orchestra.run.smithi031.stdout:2343 punch from 0x2c4e2bb to 0x2c54f14, (0x6c59 bytes) 2024-09-13T07:12:55.713 INFO:teuthology.orchestra.run.smithi031.stdout:2345 writesame 0x6eea5ca thru 0x6ef4301 data_size 0x9c(0x9d38 bytes) 2024-09-13T07:12:55.721 INFO:teuthology.orchestra.run.smithi031.stdout:2347 write 0x6f7ac4e thru 0x6f89ab0 (0xee63 bytes) 2024-09-13T07:12:55.726 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:55.727+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:55.729 INFO:teuthology.orchestra.run.smithi031.stdout:2348 read 0x3f8d0e3 thru 0x3f915f2 (0x4510 bytes) 2024-09-13T07:12:55.729 INFO:teuthology.orchestra.run.smithi031.stdout:2350 read 0x195c3d4 thru 0x19643ef (0x801c bytes) 2024-09-13T07:12:55.729 INFO:teuthology.orchestra.run.smithi031.stdout:2351 writesame 0x760920d thru 0x760c3c4 data_size 0x2b(0x31b8 bytes) 2024-09-13T07:12:55.731 INFO:teuthology.orchestra.run.smithi031.stdout:2352 write 0x7140f0b thru 0x7150220 (0xf316 bytes) 2024-09-13T07:12:55.736 INFO:teuthology.orchestra.run.smithi031.stdout:2354 trunc from 0x8dd3a1d to 0xad0c039 2024-09-13T07:12:55.742 INFO:teuthology.orchestra.run.smithi031.stdout:2355 read 0x6305fef thru 0x6309d4b (0x3d5d bytes) 2024-09-13T07:12:55.742 INFO:teuthology.orchestra.run.smithi031.stdout:2356 write 0x6047d3c thru 0x6048008 (0x2cd bytes) 2024-09-13T07:12:55.746 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:22833 3.a e38/19 3:54a6577b:::rbd_object_map.1173f6773d36:head v 38'409, mlcod=38'406)", 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.745650+0000", 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0027731710000000001, 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0027884199999999998, 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745650+0000", 2024-09-13T07:12:55.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745650+0000", 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745649+0000", 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745655+0000", 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0090000000000004e-06 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745667+0000", 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1725999999999999e-05 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745673+0000", 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4910000000000002e-06 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745698+0000", 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4916999999999999e-05 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.745715+0000", 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6484999999999999e-05 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 1 2024-09-13T07:12:55.750 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:55.751 INFO:teuthology.orchestra.run.smithi031.stdout:2359 read 0xbc9e7c thru 0xbcd129 (0x32ae bytes) 2024-09-13T07:12:55.751 INFO:teuthology.orchestra.run.smithi031.stdout:2360 clone 13 order 23 su 524288 sc 8 2024-09-13T07:12:55.756 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:55.827 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:55.827+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:55.899 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:55.899 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:55.899 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:55.899 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:55.899 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:55.907 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:55.928 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:55.928+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:914 3.14 3:2ed78eca:::rbd_data.1173f77ecad8.000000000000000a:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.763304650999999, 2024-09-13T07:12:56.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0082734499999999999, 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 914 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246421+0000", 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246420+0000", 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246424+0000", 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:12:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246430+0000", 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8000000000000001e-06 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246434+0000", 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246477+0000", 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2443e-05 2024-09-13T07:12:56.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.246632+0000", 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015543100000000001 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.254694+0000", 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080618979999999996 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.014 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1153 3.1b e25/19 3:d88532a6:::rbd_data.1173f77ecad8.000000000000015c:head v 25'14, mlcod=25'11)", 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.682157786000001, 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0091902019999999997, 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327568+0000", 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327567+0000", 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327572+0000", 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8160000000000001e-06 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327579+0000", 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.306e-06 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327585+0000", 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1279999999999996e-06 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327625+0000", 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9227999999999998e-05 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.327641+0000", 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6059999999999999e-05 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336708+0000", 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0090672949999999995 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.336758+0000", 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9870000000000002e-05 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1551 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'89, mlcod=25'86)", 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.231500483000001, 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077485289999999997, 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778225+0000", 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778224+0000", 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778233+0000", 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4689999999999997e-06 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778245+0000", 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2097e-05 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778252+0000", 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.934e-06 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778288+0000", 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6012e-05 2024-09-13T07:12:56.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.778302+0000", 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4630999999999999e-05 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785938+0000", 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076354329999999996 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.785973+0000", 2024-09-13T07:12:56.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5663000000000001e-05 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:1555 3.1d e25/19 3:b82b9b5d:::rbd_object_map.1173c53667c:head v 25'93, mlcod=25'86)", 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.229068395999999, 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0081821140000000007, 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780657+0000", 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780656+0000", 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780661+0000", 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.079e-06 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780670+0000", 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6030000000000005e-06 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780676+0000", 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780720+0000", 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3767000000000003e-05 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.780739+0000", 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9504000000000002e-05 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788759+0000", 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0080197340000000006 2024-09-13T07:12:56.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.788839+0000", 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0075999999999997e-05 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:2859 3.9 3:946400fb:::rbd_data.11734c6039c3.000000000000006d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.392040433, 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019805990999999998, 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.027 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 2859 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617685+0000", 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617683+0000", 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617695+0000", 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1626e-05 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617723+0000", 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8252999999999999e-05 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617737+0000", 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3993e-05 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617824+0000", 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6908000000000002e-05 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.617958+0000", 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013417 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.621384+0000", 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034260499999999999 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637421+0000", 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016037264999999998 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637430+0000", 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1549999999999996e-06 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637462+0000", 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1958000000000001e-05 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:40.637491+0000", 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8523000000000001e-05 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.032 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:3313 3.1a 3:5f0039c6:::rbd_data.11734c6039c3.000000000000003e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.811981605, 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.021783180999999999, 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 3313 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.033 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197744+0000", 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197742+0000", 2024-09-13T07:12:56.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197754+0000", 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2519e-05 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197797+0000", 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3028999999999997e-05 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197803+0000", 2024-09-13T07:12:56.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7420000000000003e-06 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197870+0000", 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7123000000000001e-05 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.197959+0000", 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8409e-05 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.201633+0000", 2024-09-13T07:12:56.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036746299999999999 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219455+0000", 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017821452000000002 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219464+0000", 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.8309999999999996e-06 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219498+0000", 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3948000000000001e-05 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:41.219527+0000", 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8461000000000001e-05 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4059 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:56.038 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.272481417, 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010501706, 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4059 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737244+0000", 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737245+0000", 2024-09-13T07:12:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8199999999999999e-06 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737249+0000", 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5829999999999998e-06 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.737255+0000", 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4779999999999999e-06 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.738772+0000", 2024-09-13T07:12:56.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001517853 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.738850+0000", 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7206999999999994e-05 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739214+0000", 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036488900000000002 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739761+0000", 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054667599999999995 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747700+0000", 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0079386219999999993 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747726+0000", 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5962999999999999e-05 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747746+0000", 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9899e-05 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4061 3.9 3:92d4a2e4:::rbd_data.11736f46b2f8.000000000000017d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.26977563, 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078848679999999997, 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.044 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4061 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739950+0000", 2024-09-13T07:12:56.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739949+0000", 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739954+0000", 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739968+0000", 2024-09-13T07:12:56.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3419e-05 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.739975+0000", 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3180000000000001e-06 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740015+0000", 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9450999999999999e-05 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9463999999999999e-05 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.741328+0000", 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00125416 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747778+0000", 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064493939999999998 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747786+0000", 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9060000000000002e-06 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747815+0000", 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9634000000000001e-05 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747835+0000", 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9395e-05 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.049 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:4059 3.9 e29/19 ondisk, result = 0)", 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.269651086, 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007677021, 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740074+0000", 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740077+0000", 2024-09-13T07:12:56.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740085+0000", 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3459999999999994e-06 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.740091+0000", 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5990000000000001e-06 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747692+0000", 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076006160000000001 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747697+0000", 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1560000000000003e-06 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:42.747751+0000", 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4469999999999999e-05 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4383 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.975988289, 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077768300000000002, 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4383 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033737+0000", 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033739+0000", 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6010000000000002e-06 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033743+0000", 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9709999999999998e-06 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033746+0000", 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:12:56.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035645+0000", 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001899097 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035686+0000", 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0973000000000002e-05 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.035859+0000", 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017274799999999999 2024-09-13T07:12:56.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036847+0000", 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00098787000000000007 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041442+0000", 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045950970000000002 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041477+0000", 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5629000000000001e-05 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041514+0000", 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.65e-05 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.060 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4384 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.975918096999999, 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077768220000000001, 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4384 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033807+0000", 2024-09-13T07:12:56.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033805+0000", 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033809+0000", 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1740000000000002e-06 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033814+0000", 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4479999999999996e-06 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033817+0000", 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.948e-06 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036169+0000", 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023518459999999999 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036222+0000", 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3853000000000001e-05 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036439+0000", 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021612199999999999 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037830+0000", 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0013914540000000001 2024-09-13T07:12:56.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041522+0000", 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036923220000000001 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041556+0000", 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4044000000000001e-05 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041584+0000", 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7671e-05 2024-09-13T07:12:56.066 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4385 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.975880635999999, 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078177509999999995, 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.067 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4385 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033845+0000", 2024-09-13T07:12:56.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033844+0000", 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033847+0000", 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0669999999999999e-06 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033855+0000", 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033859+0000", 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7110000000000001e-06 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036866+0000", 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030072530000000001 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.036917+0000", 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0992999999999998e-05 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037097+0000", 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018020300000000001 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.038207+0000", 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001109318 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041592+0000", 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003385047 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041631+0000", 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9196000000000002e-05 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041662+0000", 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1532000000000002e-05 2024-09-13T07:12:56.072 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:4386 3.9 3:97afc258:::rbd_object_map.11736f46b2f8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.975815528, 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078157709999999991, 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.073 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 4386 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033910+0000", 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033909+0000", 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033912+0000", 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4080000000000001e-06 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033918+0000", 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5169999999999996e-06 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.033922+0000", 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1599999999999998e-06 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037379+0000", 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034571020000000001 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037419+0000", 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0751000000000003e-05 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.037595+0000", 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017597700000000001 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.038494+0000", 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00089857399999999997 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041667+0000", 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031729610000000002 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041699+0000", 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2005000000000003e-05 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:43.041726+0000", 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6631e-05 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.079 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.7934533429999995, 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.080 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:12:56.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:12:56.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:12:56.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.086 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:7895 3.3 3:c0cb40bd:::rbd_data.117344605995.00000000000001a6:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:56.086 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:49.574210+0000", 2024-09-13T07:12:56.086 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.4355154350000001, 2024-09-13T07:12:56.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028575376, 2024-09-13T07:12:56.086 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.087 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.087 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 7895 2024-09-13T07:12:56.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574210+0000", 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574210+0000", 2024-09-13T07:12:56.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574209+0000", 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574215+0000", 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574227+0000", 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1449e-05 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574232+0000", 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3290000000000001e-06 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574271+0000", 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8724999999999999e-05 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.574355+0000", 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4098999999999996e-05 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.578913+0000", 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045581429999999997 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.602687+0000", 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.023774178999999999 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.602723+0000", 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5781000000000002e-05 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.602759+0000", 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5901000000000001e-05 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.602785+0000", 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6296999999999999e-05 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.092 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:7895 3.3 e34/19 ondisk, result = 0)", 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:49.593119+0000", 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.4166060099999997, 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0096728609999999996, 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.593119+0000", 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.593119+0000", 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.593118+0000", 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.593124+0000", 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5110000000000003e-06 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.593141+0000", 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7598999999999999e-05 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.593151+0000", 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6010000000000001e-06 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.602710+0000", 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0095595460000000004 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.602717+0000", 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.878e-06 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.602792+0000", 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4828999999999998e-05 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:7903 3.1a 3:5a9c56fe:::rbd_data.117344605995.0000000000000337:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:49.613097+0000", 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.3966286639999996, 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024198331, 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 7903 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613097+0000", 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613097+0000", 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613096+0000", 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613101+0000", 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2490000000000001e-06 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613111+0000", 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0271e-05 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613116+0000", 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1989999999999997e-06 2024-09-13T07:12:56.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613154+0000", 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7592999999999997e-05 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.613234+0000", 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0307000000000002e-05 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.616906+0000", 2024-09-13T07:12:56.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036712659999999999 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.637223+0000", 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.020317159000000001 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.637232+0000", 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6290000000000005e-06 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.637268+0000", 2024-09-13T07:12:56.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151999999999998e-05 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.637295+0000", 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7427999999999999e-05 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:7903 3.1a e34/19 ondisk, result = 0)", 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:49.628503+0000", 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 6.3812227630000002, 2024-09-13T07:12:56.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0087225689999999995, 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.628503+0000", 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.628503+0000", 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.628502+0000", 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.628508+0000", 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2380000000000002e-06 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.628526+0000", 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7691999999999999e-05 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.628535+0000", 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6679999999999996e-06 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.637208+0000", 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0086732230000000007 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.637217+0000", 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7609999999999996e-06 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:49.637225+0000", 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9680000000000006e-06 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:8952 3.1a 3:5daa5ed8:::rbd_data.11737e4ac805.00000000000000a5:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:50.613505+0000", 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3962201509999996, 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015150805999999999, 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.106 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 8952 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613505+0000", 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613505+0000", 2024-09-13T07:12:56.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613504+0000", 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613509+0000", 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613517+0000", 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5229999999999995e-06 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613522+0000", 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7400000000000004e-06 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613550+0000", 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7552e-05 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.613623+0000", 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3146999999999996e-05 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 1", 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.616086+0000", 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002463284 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.628564+0000", 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012477609000000001 2024-09-13T07:12:56.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.628575+0000", 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0732e-05 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.628621+0000", 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6480999999999998e-05 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.628656+0000", 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4965000000000002e-05 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.111 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:8953 3.14 3:2c050435:::rbd_data.11737e4ac805.00000000000000a7:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:50.615788+0000", 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3939377479999999, 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011650168000000001, 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 8953 2024-09-13T07:12:56.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615788+0000", 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615788+0000", 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615787+0000", 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615791+0000", 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6600000000000001e-06 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615799+0000", 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9209999999999992e-06 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615803+0000", 2024-09-13T07:12:56.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615833+0000", 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9611000000000002e-05 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.615928+0000", 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5001999999999996e-05 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.627351+0000", 2024-09-13T07:12:56.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011423803 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.627387+0000", 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5280000000000001e-05 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:50.627438+0000", 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0992000000000002e-05 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.116 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.117 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:56.118 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:56.118 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:56.119+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:56.219 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:56.220+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:56.262 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:12:56.263 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:56.263 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:56.263 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:56.263 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:56.274 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:56.320 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:56.321+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:56.376 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:56.376 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:56.376 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:56.376 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:56.376 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:56.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:56.421 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:56.421+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T07:12:56.468 INFO:teuthology.orchestra.run.smithi031.stdout:2362 read 0x8f97111 thru 0x8fa243e (0xb32e bytes) 2024-09-13T07:12:56.469 INFO:teuthology.orchestra.run.smithi031.stdout:2364 read 0x80a189f thru 0x80a67d8 (0x4f3a bytes) 2024-09-13T07:12:56.470 INFO:teuthology.orchestra.run.smithi031.stdout:2365 punch from 0x14a697a to 0x14a6ec1, (0x547 bytes) 2024-09-13T07:12:56.479 INFO:teuthology.orchestra.run.smithi031.stdout:2366 writesame 0xc08579e thru 0xc089a22 data_size 0x1(0x4285 bytes) 2024-09-13T07:12:56.479 INFO:teuthology.orchestra.run.smithi031.stdout:2367 read 0xb84ee4c thru 0xb8522b7 (0x346c bytes) 2024-09-13T07:12:56.479 INFO:teuthology.orchestra.run.smithi031.stdout:2368 read 0xbdb88c9 thru 0xbdc5010 (0xc748 bytes) 2024-09-13T07:12:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:2369 trunc from 0xc089a23 to 0xa15ca71 2024-09-13T07:12:56.497 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.498 INFO:teuthology.orchestra.run.smithi031.stdout:2371 read 0x7c3859f thru 0x7c40e14 (0x8876 bytes) 2024-09-13T07:12:56.498 INFO:teuthology.orchestra.run.smithi031.stdout:2373 punch from 0x8952b24 to 0x89598e2, (0x6dbe bytes) 2024-09-13T07:12:56.505 INFO:teuthology.orchestra.run.smithi031.stdout:2375 write 0xbbf7010 thru 0xbbfefbf (0x7fb0 bytes) 2024-09-13T07:12:56.506 INFO:teuthology.orchestra.run.smithi031.stdout:2376 punch from 0x31dd51 to 0x3229f6, (0x4ca5 bytes) 2024-09-13T07:12:56.506 INFO:teuthology.orchestra.run.smithi031.stdout:2377 writesame 0x9b6946c thru 0x9b6f16e data_size 0x1(0x5d03 bytes) 2024-09-13T07:12:56.514 INFO:teuthology.orchestra.run.smithi031.stdout:2378 write 0xaadbe1 thru 0xab0a4c (0x2e6c bytes) 2024-09-13T07:12:56.521 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:56.522+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:56.523 INFO:teuthology.orchestra.run.smithi031.stdout:2380 write 0x5f5a8d1 thru 0x5f5f27f (0x49af bytes) 2024-09-13T07:12:56.536 INFO:teuthology.orchestra.run.smithi031.stdout:2381 write 0x1a8d09 thru 0x1b2654 (0x994c bytes) 2024-09-13T07:12:56.539 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:56.540 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:56.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:56.540 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:56.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1309 3.1d 3:bcefd1cd:::rbd_data.1173c53667c.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:56.541 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:56.541 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.074642046000001, 2024-09-13T07:12:56.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014288309000000001, 2024-09-13T07:12:56.541 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.541 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:56.542 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.542 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.542 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.542 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1309 2024-09-13T07:12:56.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.542 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:56.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:56.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463745+0000", 2024-09-13T07:12:56.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:56.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463751+0000", 2024-09-13T07:12:56.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:12:56.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463761+0000", 2024-09-13T07:12:56.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3589999999999997e-06 2024-09-13T07:12:56.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463767+0000", 2024-09-13T07:12:56.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4420000000000003e-06 2024-09-13T07:12:56.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472731+0000", 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0089641479999999999 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472858+0000", 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012669100000000001 2024-09-13T07:12:56.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473057+0000", 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019875900000000001 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:56.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477793+0000", 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047361900000000004 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477983+0000", 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019018299999999999 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478014+0000", 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1062999999999997e-05 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478034+0000", 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.978e-05 2024-09-13T07:12:56.552 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21839 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027805+0000", 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.510583145, 2024-09-13T07:12:56.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012835941, 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21839 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027805+0000", 2024-09-13T07:12:56.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027805+0000", 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027804+0000", 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027806+0000", 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.074e-06 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027812+0000", 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027816+0000", 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9829999999999998e-06 2024-09-13T07:12:56.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040549+0000", 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012733177 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040590+0000", 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1423e-05 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040641+0000", 2024-09-13T07:12:56.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0201999999999998e-05 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21840 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027820+0000", 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5105682410000001, 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012957078, 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21840 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027820+0000", 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027820+0000", 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027819+0000", 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027821+0000", 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8989999999999999e-06 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027826+0000", 2024-09-13T07:12:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2310000000000004e-06 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027831+0000", 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1069999999999998e-06 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040661+0000", 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012830504 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040716+0000", 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5458000000000003e-05 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040777+0000", 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0183999999999998e-05 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.562 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21841 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5105533759999998, 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013061882, 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21841 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027836+0000", 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8339999999999999e-06 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027841+0000", 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4779999999999999e-06 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027846+0000", 2024-09-13T07:12:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3279999999999999e-06 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040798+0000", 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012952353999999999 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040841+0000", 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2707000000000003e-05 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040896+0000", 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5516999999999997e-05 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21842 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027850+0000", 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5105382600000001, 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013245323, 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21842 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027850+0000", 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027850+0000", 2024-09-13T07:12:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027849+0000", 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027851+0000", 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027857+0000", 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4600000000000002e-06 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027861+0000", 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3590000000000001e-06 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040924+0000", 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013063162999999999 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041004+0000", 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9406999999999994e-05 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041095+0000", 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1305999999999996e-05 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21843 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.571 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5105229410000001, 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013344300999999999, 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21843 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027867+0000", 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9879999999999999e-06 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027872+0000", 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4430000000000002e-06 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027883+0000", 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1267000000000001e-05 2024-09-13T07:12:56.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041120+0000", 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013236494999999999 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041161+0000", 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1737999999999999e-05 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041209+0000", 2024-09-13T07:12:56.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7709e-05 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21844 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5104999430000001, 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013515821000000001, 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.576 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21844 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027890+0000", 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.407e-06 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027896+0000", 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027900+0000", 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6489999999999999e-06 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041238+0000", 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013337521999999999 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041317+0000", 2024-09-13T07:12:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9258000000000006e-05 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041404+0000", 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21845 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027905+0000", 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5104832909999999, 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013605556, 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21845 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.582 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027905+0000", 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027905+0000", 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027904+0000", 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027911+0000", 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4899999999999997e-06 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027917+0000", 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027921+0000", 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041422+0000", 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013500955 2024-09-13T07:12:56.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041463+0000", 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0828000000000003e-05 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041510+0000", 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7299999999999998e-05 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21846 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027925+0000", 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.510463004, 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013726685000000001, 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.587 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21846 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027925+0000", 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027925+0000", 2024-09-13T07:12:56.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027924+0000", 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027927+0000", 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.215e-06 2024-09-13T07:12:56.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027932+0000", 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027937+0000", 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4649999999999996e-06 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041532+0000", 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013595177999999999 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041589+0000", 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7564e-05 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041652+0000", 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2218000000000006e-05 2024-09-13T07:12:56.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21847 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.510447511, 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013846261, 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21847 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027942+0000", 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0820000000000001e-06 2024-09-13T07:12:56.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027947+0000", 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027952+0000", 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2230000000000001e-06 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041683+0000", 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013731577999999999 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041733+0000", 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9669999999999997e-05 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041787+0000", 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3650000000000003e-05 2024-09-13T07:12:56.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21848 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5104325030000001, 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013953712, 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.597 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21848 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027957+0000", 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0379999999999998e-06 2024-09-13T07:12:56.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027963+0000", 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027967+0000", 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2560000000000004e-06 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041797+0000", 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013830488 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041846+0000", 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8865000000000001e-05 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041909+0000", 2024-09-13T07:12:56.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2941000000000006e-05 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21849 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027971+0000", 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5104172230000001, 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014093507, 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21849 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027971+0000", 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027971+0000", 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027970+0000", 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027972+0000", 2024-09-13T07:12:56.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0080000000000001e-06 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027978+0000", 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3890000000000004e-06 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027982+0000", 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1230000000000002e-06 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041945+0000", 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013963606999999999 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041996+0000", 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0949000000000003e-05 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042064+0000", 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.779e-05 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21850 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5104019819999999, 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014194567999999999, 2024-09-13T07:12:56.607 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21850 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:56.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027988+0000", 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027993+0000", 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4759999999999997e-06 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027997+0000", 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1910000000000002e-06 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042088+0000", 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01409063 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042131+0000", 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3296e-05 2024-09-13T07:12:56.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042180+0000", 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9212000000000002e-05 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21851 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1507328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:56.612 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5103868299999998, 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014302794000000001, 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21851 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028003+0000", 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028009+0000", 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028014+0000", 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3050000000000003e-06 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042202+0000", 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014188186 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042252+0000", 2024-09-13T07:12:56.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0096999999999998e-05 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042304+0000", 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1841000000000001e-05 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21852 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.617 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5103703529999999, 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014397584, 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21852 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028019+0000", 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9860000000000001e-06 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028024+0000", 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2530000000000004e-06 2024-09-13T07:12:56.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028029+0000", 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2980000000000002e-06 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042324+0000", 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014295560000000001 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042367+0000", 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2534999999999998e-05 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042415+0000", 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8288999999999998e-05 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21853 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.510355412, 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014507977, 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21853 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028034+0000", 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0140000000000001e-06 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028039+0000", 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3870000000000003e-06 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028044+0000", 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6e-06 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042437+0000", 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014393056 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042483+0000", 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5859999999999998e-05 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042540+0000", 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7404999999999997e-05 2024-09-13T07:12:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21854 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5103394240000001, 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014603028000000001, 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21854 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028050+0000", 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9350000000000001e-06 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028056+0000", 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028060+0000", 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4909999999999999e-06 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042561+0000", 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014501193000000001 2024-09-13T07:12:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042602+0000", 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0831000000000003e-05 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042651+0000", 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9356e-05 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21855 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1998848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5103237169999999, 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014702378, 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21855 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028066+0000", 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.995e-06 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028071+0000", 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.383e-06 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028075+0000", 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3170000000000003e-06 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042672+0000", 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014596560999999999 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042717+0000", 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4994999999999999e-05 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042767+0000", 2024-09-13T07:12:56.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9471000000000001e-05 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21856 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 2154496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028080+0000", 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5103081350000001, 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014797103000000001, 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.638 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21856 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028080+0000", 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028080+0000", 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028079+0000", 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028081+0000", 2024-09-13T07:12:56.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.871e-06 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028086+0000", 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028091+0000", 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3579999999999996e-06 2024-09-13T07:12:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042787+0000", 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014696407 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042829+0000", 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1499999999999999e-05 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042877+0000", 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8139000000000001e-05 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21857 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 2179072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.510292655, 2024-09-13T07:12:56.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014904802, 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21857 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:56.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028097+0000", 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0449999999999999e-06 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028102+0000", 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4020000000000001e-06 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:56.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028107+0000", 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2420000000000002e-06 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042898+0000", 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014791871 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042942+0000", 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3778e-05 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:56.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043000+0000", 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7819999999999999e-05 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:56.648 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:56.651 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:2385 write 0xd555f02 thru 0xd55e79c (0x889b bytes) 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:2388 write 0xb77a3f2 thru 0xb77f935 (0x5544 bytes) 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:2389 write 0x2a49d8f thru 0x2a49dff (0x71 bytes) 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:2390 writesame 0x8ecbe68 thru 0x8ed09bd data_size 0x1(0x4b56 bytes) 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:2393 trunc from 0xd55e79d to 0x40977c6 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:2394 punch from 0xf7a1cf to 0xf81e47, (0x7c78 bytes) 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:2395 punch from 0x16ae813 to 0x16b70ee, (0x88db bytes) 2024-09-13T07:12:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:2396 write 0x27e9292 thru 0x27ef837 (0x65a6 bytes) 2024-09-13T07:12:56.653 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:56.649+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:2397 read 0x371b18e thru 0x372ab9a (0xfa0d bytes) 2024-09-13T07:12:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:2399 write 0x141c969 thru 0x141d110 (0x7a8 bytes) 2024-09-13T07:12:56.685 INFO:teuthology.orchestra.run.smithi031.stdout:2400 writesame 0x6090682 thru 0x609492c data_size 0x1(0x42ab bytes) 2024-09-13T07:12:56.686 INFO:teuthology.orchestra.run.smithi031.stdout:2401 write 0xd1c552 thru 0xd2209f (0x5b4e bytes) 2024-09-13T07:12:56.709 INFO:teuthology.orchestra.run.smithi031.stdout:2402 trunc from 0x609492d to 0x17a0fce 2024-09-13T07:12:56.732 INFO:teuthology.orchestra.run.smithi031.stdout:2404 read 0x821aef thru 0x82fd7c (0xe28e bytes) 2024-09-13T07:12:56.739 INFO:teuthology.orchestra.run.smithi031.stdout:2405 trunc from 0x17a0fce to 0x6d9a0d2 2024-09-13T07:12:56.745 INFO:teuthology.orchestra.run.smithi031.stdout:2406 read 0x5031747 thru 0x503b20c (0x9ac6 bytes) 2024-09-13T07:12:56.745 INFO:teuthology.orchestra.run.smithi031.stdout:2407 writesame 0x248189a thru 0x248666c data_size 0x3(0x4dd3 bytes) 2024-09-13T07:12:56.748 INFO:teuthology.orchestra.run.smithi031.stdout:2408 read 0x1003321 thru 0x1005c3c (0x291c bytes) 2024-09-13T07:12:56.750 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:56.751+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:56.758 INFO:teuthology.orchestra.run.smithi031.stdout:2410 write 0xa5afd68 thru 0xa5b2d29 (0x2fc2 bytes) 2024-09-13T07:12:56.760 INFO:teuthology.orchestra.run.smithi031.stdout:2414 read 0x8fef5a8 thru 0x8ff0c5c (0x16b5 bytes) 2024-09-13T07:12:56.760 INFO:teuthology.orchestra.run.smithi031.stdout:2415 read 0x20a481e thru 0x20af13f (0xa922 bytes) 2024-09-13T07:12:56.761 INFO:teuthology.orchestra.run.smithi031.stdout:2416 read 0x2c1ab9f thru 0x2c21064 (0x64c6 bytes) 2024-09-13T07:12:56.769 INFO:teuthology.orchestra.run.smithi031.stdout:2417 write 0xd20cce2 thru 0xd210152 (0x3471 bytes) 2024-09-13T07:12:56.773 INFO:teuthology.orchestra.run.smithi031.stdout:2418 trunc from 0xd210153 to 0xc6b2468 2024-09-13T07:12:56.779 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.779 INFO:teuthology.orchestra.run.smithi031.stdout:2420 punch from 0x5c02d18 to 0x5c04a38, (0x1d20 bytes) 2024-09-13T07:12:56.780 INFO:teuthology.orchestra.run.smithi031.stdout:2422 read 0xb9dc878 thru 0xb9e0c8a (0x4413 bytes) 2024-09-13T07:12:56.780 INFO:teuthology.orchestra.run.smithi031.stdout:2423 write 0x4bf62c7 thru 0x4bff7c8 (0x9502 bytes) 2024-09-13T07:12:56.782 INFO:teuthology.orchestra.run.smithi031.stdout:2424 punch from 0x6d53e13 to 0x6d58534, (0x4721 bytes) 2024-09-13T07:12:56.783 INFO:teuthology.orchestra.run.smithi031.stdout:2427 trunc from 0xc6b2468 to 0xb747e21 2024-09-13T07:12:56.793 INFO:teuthology.orchestra.run.smithi031.stdout:2428 read 0x7a1dec2 thru 0x7a27b36 (0x9c75 bytes) 2024-09-13T07:12:56.795 INFO:teuthology.orchestra.run.smithi031.stdout:2429 trunc from 0xb747e21 to 0xd816af5 2024-09-13T07:12:56.797 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:56.798 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:56.798 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:12:56.798 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:56.800 INFO:teuthology.orchestra.run.smithi031.stdout:2430 read 0x8209924 thru 0x821128b (0x7968 bytes) 2024-09-13T07:12:56.800 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.800 INFO:teuthology.orchestra.run.smithi031.stdout:2432 read 0x10f485 thru 0x11daf0 (0xe66c bytes) 2024-09-13T07:12:56.800 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.801 INFO:teuthology.orchestra.run.smithi031.stdout:2434 read 0xb930176 thru 0xb94009c (0xff27 bytes) 2024-09-13T07:12:56.801 INFO:teuthology.orchestra.run.smithi031.stdout:2435 write 0x7cbd2d9 thru 0x7cc6e2a (0x9b52 bytes) 2024-09-13T07:12:56.803 INFO:teuthology.orchestra.run.smithi031.stdout:2437 writesame 0x880eba6 thru 0x88183a6 data_size 0x33(0x9801 bytes) 2024-09-13T07:12:56.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.806 INFO:teuthology.orchestra.run.smithi031.stdout:2440 read 0x323bc58 thru 0x3241389 (0x5732 bytes) 2024-09-13T07:12:56.807 INFO:teuthology.orchestra.run.smithi031.stdout:2441 read 0x9e0a5e2 thru 0x9e15a20 (0xb43f bytes) 2024-09-13T07:12:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:2442 punch from 0x459edb0 to 0x45a7447, (0x8697 bytes) 2024-09-13T07:12:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:2443 punch from 0x815caba to 0x8162c8d, (0x61d3 bytes) 2024-09-13T07:12:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:2444 punch from 0x58d9a8f to 0x58da943, (0xeb4 bytes) 2024-09-13T07:12:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:2445 write 0x87ff5e7 thru 0x880b3ae (0xbdc8 bytes) 2024-09-13T07:12:56.808 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:12:56.810 INFO:teuthology.orchestra.run.smithi031.stdout:2446 read 0x55c77a6 thru 0x55c8bd4 (0x142f bytes) 2024-09-13T07:12:56.810 INFO:teuthology.orchestra.run.smithi031.stdout:2449 punch from 0x1cc54e4 to 0x1ccff94, (0xaab0 bytes) 2024-09-13T07:12:56.810 INFO:teuthology.orchestra.run.smithi031.stdout:2450 writesame 0x724c32c thru 0x7255372 data_size 0x5(0x9047 bytes) 2024-09-13T07:12:56.814 INFO:teuthology.orchestra.run.smithi031.stdout:2451 write 0xcecfb10 thru 0xced6c57 (0x7148 bytes) 2024-09-13T07:12:56.817 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:56.817 INFO:teuthology.orchestra.run.smithi031.stdout:2453 writesame 0x1e6ad51 thru 0x1e75460 data_size 0xb0(0xa710 bytes) 2024-09-13T07:12:56.818 INFO:teuthology.orchestra.run.smithi031.stdout:2456 write 0x87a5ad0 thru 0x87b2e5a (0xd38b bytes) 2024-09-13T07:12:56.819 INFO:teuthology.orchestra.run.smithi031.stdout:2459 read 0xccabab thru 0xccbc4e (0x10a4 bytes) 2024-09-13T07:12:56.819 INFO:teuthology.orchestra.run.smithi031.stdout:2461 read 0xd57afec thru 0xd57b06d (0x82 bytes) 2024-09-13T07:12:56.826 INFO:teuthology.orchestra.run.smithi031.stdout:2462 trunc from 0xd816af5 to 0x7373f67 2024-09-13T07:12:56.837 INFO:teuthology.orchestra.run.smithi031.stdout:2463 read 0x3ca719c thru 0x3cb527c (0xe0e1 bytes) 2024-09-13T07:12:56.838 INFO:teuthology.orchestra.run.smithi031.stdout:2464 trunc from 0x7373f67 to 0x5ddf54b 2024-09-13T07:12:56.845 INFO:teuthology.orchestra.run.smithi031.stdout:2465 punch from 0x4151d88 to 0x4159319, (0x7591 bytes) 2024-09-13T07:12:56.845 INFO:teuthology.orchestra.run.smithi031.stdout:2466 writesame 0x5c1bc2f thru 0x5c25fcf data_size 0x1(0xa3a1 bytes) 2024-09-13T07:12:56.851 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:56.851+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:56.875 INFO:teuthology.orchestra.run.smithi031.stdout:2467 write 0xdff9b5f thru 0xdffb193 (0x1635 bytes) 2024-09-13T07:12:56.877 INFO:teuthology.orchestra.run.smithi031.stdout:2468 write 0x1c56959 thru 0x1c5e621 (0x7cc9 bytes) 2024-09-13T07:12:56.885 INFO:teuthology.orchestra.run.smithi031.stdout:2469 writesame 0xecb5d5d thru 0xecbb75a data_size 0x1(0x59fe bytes) 2024-09-13T07:12:56.885 INFO:teuthology.orchestra.run.smithi031.stdout:2470 punch from 0x3014d26 to 0x3020d08, (0xbfe2 bytes) 2024-09-13T07:12:56.885 INFO:teuthology.orchestra.run.smithi031.stdout:2471 write 0x655c644 thru 0x6565cea (0x96a7 bytes) 2024-09-13T07:12:56.904 INFO:teuthology.orchestra.run.smithi031.stdout:2472 trunc from 0xecbb75b to 0xee480f 2024-09-13T07:12:56.926 INFO:teuthology.orchestra.run.smithi031.stdout:2474 write 0x405bff1 thru 0x405d2c7 (0x12d7 bytes) 2024-09-13T07:12:56.942 INFO:teuthology.orchestra.run.smithi031.stdout:2475 write 0xb6c29c8 thru 0xb6cfe46 (0xd47f bytes) 2024-09-13T07:12:56.945 INFO:teuthology.orchestra.run.smithi031.stdout:2476 read 0x4c134ee thru 0x4c1a45f (0x6f72 bytes) 2024-09-13T07:12:56.945 INFO:teuthology.orchestra.run.smithi031.stdout:2477 writesame 0x7fada62 thru 0x7fb97ca data_size 0x7(0xbd69 bytes) 2024-09-13T07:12:56.950 INFO:teuthology.orchestra.run.smithi031.stdout:2478 punch from 0x1e27f4c to 0x1e35412, (0xd4c6 bytes) 2024-09-13T07:12:56.951 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:56.951+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:56.954 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:56.954 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:56.954 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:56.954 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:56.954 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:56.962 INFO:teuthology.orchestra.run.smithi031.stdout:2480 trunc from 0xb6cfe47 to 0x138351c 2024-09-13T07:12:56.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:12:56.993 INFO:teuthology.orchestra.run.smithi031.stdout:2481 write 0xcf4820b thru 0xcf51b30 (0x9926 bytes) 2024-09-13T07:12:56.996 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.001 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-13T07:12:57.001 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:12:57.002 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T07:12:57.002 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:12:57.002 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:12:57.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T07:12:57.010 INFO:teuthology.orchestra.run.smithi031.stdout:2483 trunc from 0xcf51b31 to 0x6ae534 2024-09-13T07:12:57.023 INFO:teuthology.orchestra.run.smithi031.stdout:2485 read 0x1b0b99 thru 0x1b7c50 (0x70b8 bytes) 2024-09-13T07:12:57.030 INFO:teuthology.orchestra.run.smithi031.stdout:2486 trunc from 0x6ae534 to 0x5f47c29 2024-09-13T07:12:57.038 INFO:teuthology.orchestra.run.smithi031.stdout:2487 trunc from 0x5f47c29 to 0x3a34def 2024-09-13T07:12:57.052 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:57.053+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:57.053 INFO:teuthology.orchestra.run.smithi031.stdout:2488 writesame 0x4777cca thru 0x4778be2 data_size 0x1(0xf19 bytes) 2024-09-13T07:12:57.053 INFO:teuthology.orchestra.run.smithi031.stdout:2489 read 0x158dcb2 thru 0x159ab11 (0xce60 bytes) 2024-09-13T07:12:57.054 INFO:teuthology.orchestra.run.smithi031.stdout:2493 punch from 0x7cb8fd to 0x7cec7f, (0x3382 bytes) 2024-09-13T07:12:57.054 INFO:teuthology.orchestra.run.smithi031.stdout:2497 write 0x1ab020e thru 0x1ab22c9 (0x20bc bytes) 2024-09-13T07:12:57.055 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.055 INFO:teuthology.orchestra.run.smithi031.stdout:2499 write 0x2a7ca15 thru 0x2a84fee (0x85da bytes) 2024-09-13T07:12:57.056 INFO:teuthology.orchestra.run.smithi031.stdout:2500 write 0x159095c thru 0x1590b6c (0x211 bytes) 2024-09-13T07:12:57.057 INFO:teuthology.orchestra.run.smithi031.stdout:2501 write 0x38c8d1a thru 0x38d379f (0xaa86 bytes) 2024-09-13T07:12:57.058 INFO:teuthology.orchestra.run.smithi031.stdout:2502 read 0x2a17df9 thru 0x2a1b80d (0x3a15 bytes) 2024-09-13T07:12:57.058 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:12:57.070 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:57.070 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033126360000001, 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.071 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:12:57.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1768 3.a e25/19 ondisk, result = 0)", 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.018253274999999, 2024-09-13T07:12:57.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010131984, 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052064+0000", 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052072+0000", 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9529999999999999e-06 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052090+0000", 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8263000000000001e-05 2024-09-13T07:12:57.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052102+0000", 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2078e-05 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062047+0000", 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0099446580000000003 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062059+0000", 2024-09-13T07:12:57.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2455e-05 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062197+0000", 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013813200000000001 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21398 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.061591339, 2024-09-13T07:12:57.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088116559999999993, 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21398 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:12:57.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008729+0000", 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.798e-06 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008734+0000", 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.763e-06 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008738+0000", 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7999999999999998e-06 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011757+0000", 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030188580000000001 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011838+0000", 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1021999999999998e-05 2024-09-13T07:12:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012183+0000", 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034440499999999999 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013964+0000", 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001781071 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017492+0000", 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035283329999999998 2024-09-13T07:12:57.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017520+0000", 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7687000000000001e-05 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017539+0000", 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9126e-05 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21399 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.0615761689999998, 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088642229999999992, 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21399 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008745+0000", 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008750+0000", 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008755+0000", 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6480000000000002e-06 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012580+0000", 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003825214 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012630+0000", 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9772999999999999e-05 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012831+0000", 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000201294 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013967+0000", 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001136176 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017566+0000", 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035988190000000001 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017590+0000", 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3490999999999999e-05 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017607+0000", 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.093 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21921 3.10 3:0f55d758:::rbd_data.1173f6773d36.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 2.0334418780000001, 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065309330000000001, 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21921 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:12:57.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036876+0000", 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038933+0000", 2024-09-13T07:12:57.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020573900000000001 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038947+0000", 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4071e-05 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038955+0000", 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1839999999999999e-06 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038999+0000", 2024-09-13T07:12:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3399000000000002e-05 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039104+0000", 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104798 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039324+0000", 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00022014499999999999 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041941+0000", 2024-09-13T07:12:57.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026169420000000001 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043354+0000", 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001412792 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043385+0000", 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1767999999999998e-05 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043408+0000", 2024-09-13T07:12:57.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2422999999999999e-05 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22131 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.8923608509999998, 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076826359999999996, 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.099 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22131 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:12:57.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177957+0000", 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177960+0000", 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9079999999999999e-06 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177967+0000", 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4219999999999997e-06 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177970+0000", 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9380000000000001e-06 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178788+0000", 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081823400000000004 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178835+0000", 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7348000000000001e-05 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179052+0000", 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021655000000000001 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.182995+0000", 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039430350000000001 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185592+0000", 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025971280000000002 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185618+0000", 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6390999999999999e-05 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185640+0000", 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2144e-05 2024-09-13T07:12:57.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22132 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.892332211, 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076953339999999999, 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22132 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177989+0000", 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6749999999999998e-06 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177995+0000", 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8839999999999999e-06 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177999+0000", 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6650000000000002e-06 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179333+0000", 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001334029 2024-09-13T07:12:57.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179381+0000", 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8115000000000003e-05 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179582+0000", 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020005199999999999 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183028+0000", 2024-09-13T07:12:57.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003446185 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185646+0000", 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00261845 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185666+0000", 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0301000000000001e-05 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185682+0000", 2024-09-13T07:12:57.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5333999999999999e-05 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22133 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.892315295, 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007715502, 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.111 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22133 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178005+0000", 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.745e-06 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178008+0000", 2024-09-13T07:12:57.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.354e-06 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178011+0000", 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.847e-06 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179904+0000", 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001892828 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179977+0000", 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2959000000000004e-05 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180170+0000", 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019334699999999999 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183047+0000", 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028766690000000001 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185685+0000", 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002638315 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185704+0000", 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8929000000000001e-05 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185719+0000", 2024-09-13T07:12:57.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4772e-05 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22134 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.8923051279999998, 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077416129999999996, 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22134 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:12:57.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178013+0000", 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178015+0000", 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.449e-06 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178018+0000", 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0970000000000002e-06 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178021+0000", 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9790000000000002e-06 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180446+0000", 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002425207 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180494+0000", 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7870000000000001e-05 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180680+0000", 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018602899999999999 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183074+0000", 2024-09-13T07:12:57.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023943660000000002 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185722+0000", 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026480010000000001 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185741+0000", 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8328000000000001e-05 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185755+0000", 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4552999999999999e-05 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22135 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.8922958859999999, 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007772101, 2024-09-13T07:12:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22135 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178024+0000", 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3909999999999999e-06 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178027+0000", 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178029+0000", 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4679999999999999e-06 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180968+0000", 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029389149999999998 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181015+0000", 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7178999999999997e-05 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181200+0000", 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018424700000000001 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183091+0000", 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001891413 2024-09-13T07:12:57.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185758+0000", 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026670819999999999 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185778+0000", 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9656e-05 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185795+0000", 2024-09-13T07:12:57.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7020999999999998e-05 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22384 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.675957342, 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078826639999999993, 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22384 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394363+0000", 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.511e-06 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394368+0000", 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4589999999999997e-06 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394371+0000", 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397357+0000", 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029860659999999999 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397406+0000", 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8982e-05 2024-09-13T07:12:57.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397659+0000", 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025325 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.401969+0000", 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043103020000000002 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402177+0000", 2024-09-13T07:12:57.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020735 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402215+0000", 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8532999999999999e-05 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402244+0000", 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8756999999999999e-05 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22563 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.5282571379999998, 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065088769999999997, 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22563 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542061+0000", 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542063+0000", 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4810000000000001e-06 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542066+0000", 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.954e-06 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542068+0000", 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6280000000000001e-06 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546580+0000", 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045117259999999998 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546648+0000", 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7398999999999996e-05 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546893+0000", 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024524099999999999 2024-09-13T07:12:57.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548346+0000", 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001453153 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548486+0000", 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013988499999999999 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548532+0000", 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6675000000000003e-05 2024-09-13T07:12:57.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548570+0000", 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7938000000000002e-05 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22607 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.494692184, 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067359300000000002, 2024-09-13T07:12:57.140 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22607 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575629+0000", 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.943e-06 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575635+0000", 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0249999999999999e-06 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575640+0000", 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0900000000000004e-06 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.578914+0000", 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032737140000000001 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579018+0000", 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000103928 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579262+0000", 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000243988 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.581291+0000", 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020291419999999998 2024-09-13T07:12:57.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582314+0000", 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010232660000000001 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582343+0000", 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8844e-05 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582362+0000", 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9409e-05 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22608 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.4946744350000001, 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097116479999999998, 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.146 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22608 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:12:57.147 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575646+0000", 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.148 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575655+0000", 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3359999999999995e-06 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575660+0000", 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579853+0000", 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004193123 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.149 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579939+0000", 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6316999999999994e-05 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580288+0000", 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034927800000000001 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582212+0000", 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019236679999999999 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585304+0000", 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030922509999999999 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585337+0000", 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585356+0000", 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8899e-05 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.151 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22609 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.4946473789999999, 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097598709999999998, 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22609 2024-09-13T07:12:57.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575674+0000", 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0850000000000001e-06 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575685+0000", 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195e-05 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575690+0000", 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3589999999999999e-06 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580716+0000", 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050255289999999999 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580764+0000", 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8402000000000002e-05 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580983+0000", 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021850999999999999 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585136+0000", 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041530070000000002 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585385+0000", 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024898500000000002 2024-09-13T07:12:57.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585412+0000", 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7240999999999999e-05 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585431+0000", 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8995e-05 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22684 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.440350963, 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069291209999999999, 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22684 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.158 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629967+0000", 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.159 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629969+0000", 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.468e-06 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629972+0000", 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0520000000000002e-06 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629975+0000", 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.160 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634438+0000", 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044633980000000004 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634515+0000", 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7020000000000005e-05 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634768+0000", 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025338500000000001 2024-09-13T07:12:57.161 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636038+0000", 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012701800000000001 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636808+0000", 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076938199999999999 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.162 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636854+0000", 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6149000000000001e-05 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636897+0000", 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2796999999999997e-05 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22685 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:12:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.4403420790000001, 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072232559999999999, 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22685 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.164 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629976+0000", 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629978+0000", 2024-09-13T07:12:57.165 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.42e-06 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629981+0000", 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7630000000000001e-06 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629983+0000", 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.424e-06 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635281+0000", 2024-09-13T07:12:57.166 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052976969999999996 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635373+0000", 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2596000000000006e-05 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635650+0000", 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027690599999999999 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636553+0000", 2024-09-13T07:12:57.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000902646 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637134+0000", 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058095900000000003 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637169+0000", 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4793999999999999e-05 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637200+0000", 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1254999999999998e-05 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22783 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.355319377, 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062902130000000002, 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22783 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715002+0000", 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8430000000000001e-06 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715009+0000", 2024-09-13T07:12:57.171 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5379999999999997e-06 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715014+0000", 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.719299+0000", 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042849819999999997 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.172 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.719381+0000", 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2181999999999993e-05 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.719740+0000", 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035867400000000002 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721214+0000", 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001474508 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.173 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721223+0000", 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5010000000000004e-06 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721267+0000", 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4359e-05 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721289+0000", 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2585000000000001e-05 2024-09-13T07:12:57.174 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22784 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.3553005970000001, 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072328160000000004, 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22784 2024-09-13T07:12:57.175 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.176 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715020+0000", 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715026+0000", 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9190000000000004e-06 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.177 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715031+0000", 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5889999999999996e-06 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720306+0000", 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005275475 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720370+0000", 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3802999999999999e-05 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720620+0000", 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025008499999999999 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721297+0000", 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00067708199999999997 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722163+0000", 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00086599500000000005 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.179 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722217+0000", 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3393999999999997e-05 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722251+0000", 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4223000000000001e-05 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22798 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:57.180 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.34680545, 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064185800000000001, 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22798 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:12:57.181 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723516+0000", 2024-09-13T07:12:57.182 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.208e-06 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723524+0000", 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3720000000000005e-06 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723531+0000", 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3339999999999998e-06 2024-09-13T07:12:57.183 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.727445+0000", 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039143729999999996 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.727528+0000", 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3009000000000004e-05 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.727889+0000", 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036129800000000002 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.728636+0000", 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00074646599999999999 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.729858+0000", 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001222239 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.729911+0000", 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3397999999999999e-05 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.729932+0000", 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0387e-05 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:57.186 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:57.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2504 write 0xa68c16c thru 0xa696492 (0xa327 bytes) 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2505 read 0x701e24f thru 0x70230fb (0x4ead bytes) 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2506 writesame 0x3eb06f3 thru 0x3eb2927 data_size 0x15(0x2235 bytes) 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2508 read 0x3a071e thru 0x3a1389 (0xc6c bytes) 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2510 punch from 0x2ceed36 to 0x2cfafa3, (0xc26d bytes) 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2511 writesame 0x63aeb6 thru 0x63b502 data_size 0x1(0x64d bytes) 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2513 write 0x929a84d thru 0x929da24 (0x31d8 bytes) 2024-09-13T07:12:57.190 INFO:teuthology.orchestra.run.smithi031.stdout:2514 write 0xf09a3a thru 0xf09cb8 (0x27f bytes) 2024-09-13T07:12:57.191 INFO:teuthology.orchestra.run.smithi031.stdout:2516 trunc from 0xa696493 to 0x153843f 2024-09-13T07:12:57.191 INFO:teuthology.orchestra.run.smithi031.stdout:2517 writesame 0x2335bd9 thru 0x2337087 data_size 0x5(0x14af bytes) 2024-09-13T07:12:57.191 INFO:teuthology.orchestra.run.smithi031.stdout:2518 write 0x4e50f91 thru 0x4e51c67 (0xcd7 bytes) 2024-09-13T07:12:57.191 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.191 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:12:57.191 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "" 2024-09-13T07:12:57.191 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:12:57.192 INFO:teuthology.orchestra.run.smithi031.stdout:2524 write 0xb105667 thru 0xb113c71 (0xe60b bytes) 2024-09-13T07:12:57.192 INFO:teuthology.orchestra.run.smithi031.stdout:2525 write 0x8452863 thru 0x8453caa (0x1448 bytes) 2024-09-13T07:12:57.193 INFO:teuthology.orchestra.run.smithi031.stdout:2526 write 0xcdf0262 thru 0xcdfa3c9 (0xa168 bytes) 2024-09-13T07:12:57.193 INFO:teuthology.orchestra.run.smithi031.stdout:2527 read 0x238c41a thru 0x239a5ca (0xe1b1 bytes) 2024-09-13T07:12:57.193 INFO:teuthology.orchestra.run.smithi031.stdout:2532 clone 14 order 21 su 65536 sc 7 2024-09-13T07:12:57.193 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:57.187+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:57.208 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:57.209 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:57.225 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:57.242 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:57.258 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:57.258 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:57.276 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:57.289 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:57.289+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:57.296 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:57.296 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:57.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:57.321 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:57.321 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:12:57.325 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:12:57.325 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:12:57.327 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone13 from 0xcdfa3ca (overlap 0x6ae534) to 0x4ad3a4 2024-09-13T07:12:57.329 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:57.329 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:57.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:12:57.389 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T07:12:57.389 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:57.390+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:57.493 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:57.493+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:57.510 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:57.510 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:57.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:12:57.588 INFO:teuthology.orchestra.run.smithi031.stdout:2533 trunc from 0xcdfa3ca to 0xcd44013 2024-09-13T07:12:57.594 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:57.594+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:57.610 INFO:teuthology.orchestra.run.smithi031.stdout:2534 trunc from 0xcd44013 to 0xabd3cc4 2024-09-13T07:12:57.631 INFO:teuthology.orchestra.run.smithi031.stdout:2535 read 0xbbf367 thru 0xbc0a2d (0x16c7 bytes) 2024-09-13T07:12:57.633 INFO:teuthology.orchestra.run.smithi031.stdout:2536 trunc from 0xabd3cc4 to 0x98c4913 2024-09-13T07:12:57.669 INFO:teuthology.orchestra.run.smithi031.stdout:2538 writesame 0xc434449 thru 0xc435b27 data_size 0x1(0x16df bytes) 2024-09-13T07:12:57.669 INFO:teuthology.orchestra.run.smithi031.stdout:2540 write 0x524a1b9 thru 0x524dd4f (0x3b97 bytes) 2024-09-13T07:12:57.672 INFO:teuthology.orchestra.run.smithi031.stdout:2541 read 0x658921 thru 0x66308f (0xa76f bytes) 2024-09-13T07:12:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:2542 write 0xeafa940 thru 0xeb07319 (0xc9da bytes) 2024-09-13T07:12:57.684 INFO:teuthology.orchestra.run.smithi031.stdout:2545 writesame 0x4a07323 thru 0x4a0aedf data_size 0x1(0x3bbd bytes) 2024-09-13T07:12:57.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:12:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:2546 punch from 0xaeedd4d to 0xaef0d27, (0x2fda bytes) 2024-09-13T07:12:57.693 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:2549 read 0x6963d6b thru 0x6970941 (0xcbd7 bytes) 2024-09-13T07:12:57.694 INFO:teuthology.orchestra.run.smithi031.stdout:2551 write 0x4d5dba3 thru 0x4d63c9b (0x60f9 bytes) 2024-09-13T07:12:57.695 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:57.695+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:2552 write 0x4b3f4fe thru 0x4b445a4 (0x50a7 bytes) 2024-09-13T07:12:57.700 INFO:teuthology.orchestra.run.smithi031.stdout:2553 trunc from 0xeb0731a to 0xe278a74 2024-09-13T07:12:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:2554 read 0xb999ade thru 0xb9a4347 (0xa86a bytes) 2024-09-13T07:12:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:2555 write 0x35f461f thru 0x35f8506 (0x3ee8 bytes) 2024-09-13T07:12:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:2556 read 0x76a5a87 thru 0x76af899 (0x9e13 bytes) 2024-09-13T07:12:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:2558 read 0xb23956f thru 0xb243ffd (0xaa8f bytes) 2024-09-13T07:12:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:2560 write 0x70599b5 thru 0x70619c0 (0x800c bytes) 2024-09-13T07:12:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:2561 read 0xcffa692 thru 0xcfffa79 (0x53e8 bytes) 2024-09-13T07:12:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:2562 read 0xc34ae62 thru 0xc34e315 (0x34b4 bytes) 2024-09-13T07:12:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:2565 punch from 0x917228a to 0x917716e, (0x4ee4 bytes) 2024-09-13T07:12:57.714 INFO:teuthology.orchestra.run.smithi031.stdout:2566 punch from 0x3f74da4 to 0x3f7ad24, (0x5f80 bytes) 2024-09-13T07:12:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:2567 trunc from 0xe278a74 to 0x1dfe452 2024-09-13T07:12:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.751 INFO:teuthology.orchestra.run.smithi031.stdout:2569 read 0x1bf7097 thru 0x1c04806 (0xd770 bytes) 2024-09-13T07:12:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:2570 writesame 0x8e41405 thru 0x8e45c13 data_size 0x2b(0x480f bytes) 2024-09-13T07:12:57.764 INFO:teuthology.orchestra.run.smithi031.stdout:2571 punch from 0x7b3f82c to 0x7b46da6, (0x757a bytes) 2024-09-13T07:12:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:2572 writesame 0xa4d58ff thru 0xa4daf69 data_size 0x1(0x566b bytes) 2024-09-13T07:12:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:2574 read 0x433e5c2 thru 0x4341be0 (0x361f bytes) 2024-09-13T07:12:57.770 INFO:teuthology.orchestra.run.smithi031.stdout:2576 punch from 0x6be29f0 to 0x6be72ed, (0x48fd bytes) 2024-09-13T07:12:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:2577 read 0x495939c thru 0x495ead7 (0x573c bytes) 2024-09-13T07:12:57.771 INFO:teuthology.orchestra.run.smithi031.stdout:2579 read 0x9d7d79b thru 0x9d81ca5 (0x450b bytes) 2024-09-13T07:12:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:2581 trunc from 0xa4daf6a to 0x400fc36 2024-09-13T07:12:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:2582 punch from 0x32c5118 to 0x32cd8cf, (0x87b7 bytes) 2024-09-13T07:12:57.787 INFO:teuthology.orchestra.run.smithi031.stdout:2584 trunc from 0x400fc36 to 0x870fe7f 2024-09-13T07:12:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:2585 write 0x4fb2730 thru 0x4fb30cf (0x9a0 bytes) 2024-09-13T07:12:57.793 INFO:teuthology.orchestra.run.smithi031.stdout:2587 write 0x8528f2b thru 0x852e791 (0x5867 bytes) 2024-09-13T07:12:57.795 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:57.796+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:2588 read 0x46fd11b thru 0x4706098 (0x8f7e bytes) 2024-09-13T07:12:57.807 INFO:teuthology.orchestra.run.smithi031.stdout:2589 write 0xc62b324 thru 0xc63a47d (0xf15a bytes) 2024-09-13T07:12:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:57.818 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:57.819 INFO:teuthology.orchestra.run.smithi031.stdout:2590 trunc from 0xc63a47e to 0x3ec4465 2024-09-13T07:12:57.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:12:57.836 INFO:teuthology.orchestra.run.smithi031.stdout:2591 trunc from 0x3ec4465 to 0xb37d4b4 2024-09-13T07:12:57.840 INFO:teuthology.orchestra.run.smithi031.stdout:2593 writesame 0x25bf3be thru 0x25cc023 data_size 0x72(0xcc66 bytes) 2024-09-13T07:12:57.842 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.843 INFO:teuthology.orchestra.run.smithi031.stdout:2598 read 0x9af997c thru 0x9afdd8d (0x4412 bytes) 2024-09-13T07:12:57.843 INFO:teuthology.orchestra.run.smithi031.stdout:2599 read 0x19dd8cf thru 0x19eb11d (0xd84f bytes) 2024-09-13T07:12:57.843 INFO:teuthology.orchestra.run.smithi031.stdout:2603 read 0x82cceb6 thru 0x82db1b0 (0xe2fb bytes) 2024-09-13T07:12:57.854 INFO:teuthology.orchestra.run.smithi031.stdout:2604 trunc from 0xb37d4b4 to 0x15348be 2024-09-13T07:12:57.875 INFO:teuthology.orchestra.run.smithi031.stdout:2605 read 0xee5e1e thru 0xeed693 (0x7876 bytes) 2024-09-13T07:12:57.893 INFO:teuthology.orchestra.run.smithi031.stdout:2606 writesame 0xc62d8d7 thru 0xc6326e8 data_size 0x3(0x4e12 bytes) 2024-09-13T07:12:57.893 INFO:teuthology.orchestra.run.smithi031.stdout:2607 trunc from 0xc6326e9 to 0xd26ee06 2024-09-13T07:12:57.896 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:57.896+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:57.898 INFO:teuthology.orchestra.run.smithi031.stdout:2608 read 0xcd0c65c thru 0xcd197cd (0xd172 bytes) 2024-09-13T07:12:57.898 INFO:teuthology.orchestra.run.smithi031.stdout:2612 punch from 0x5ceca06 to 0x5cfb77b, (0xed75 bytes) 2024-09-13T07:12:57.898 INFO:teuthology.orchestra.run.smithi031.stdout:2613 punch from 0x14b1d59 to 0x14b43dc, (0x2683 bytes) 2024-09-13T07:12:57.898 INFO:teuthology.orchestra.run.smithi031.stdout:2614 write 0x57137ec thru 0x571c35d (0x8b72 bytes) 2024-09-13T07:12:57.900 INFO:teuthology.orchestra.run.smithi031.stdout:2615 read 0x6dce3ab thru 0x6dd80b0 (0x9d06 bytes) 2024-09-13T07:12:57.901 INFO:teuthology.orchestra.run.smithi031.stdout:2616 write 0x62aa3fc thru 0x62b97b6 (0xf3bb bytes) 2024-09-13T07:12:57.904 INFO:teuthology.orchestra.run.smithi031.stdout:2618 writesame 0x6a19c4c thru 0x6a2086a data_size 0x1(0x6c1f bytes) 2024-09-13T07:12:57.913 INFO:teuthology.orchestra.run.smithi031.stdout:2619 write 0x9fb11b thru 0xa01cf6 (0x6bdc bytes) 2024-09-13T07:12:57.920 INFO:teuthology.orchestra.run.smithi031.stdout:2620 read 0x77f8e81 thru 0x77fa2b0 (0x1430 bytes) 2024-09-13T07:12:57.926 INFO:teuthology.orchestra.run.smithi031.stdout:2621 write 0xe57740f thru 0xe5843b0 (0xcfa2 bytes) 2024-09-13T07:12:57.929 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:57.933 INFO:teuthology.orchestra.run.smithi031.stdout:2625 trunc from 0xe5843b1 to 0xac050e2 2024-09-13T07:12:57.941 INFO:teuthology.orchestra.run.smithi031.stdout:2626 clone 15 order 19 su 4096 sc 4 2024-09-13T07:12:57.958 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:57.958 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:57.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:12:58.008 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:58.008+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:58.099 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:58.100+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:58.113 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:58.114 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:58.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:12:58.200 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:58.201+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:58.228 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:58.229 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:58.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:12:58.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:12:58.325 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:58.325+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:58.337 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone14 from 0xac050e2 (overlap 0x15348be) to 0x87b6b2 2024-09-13T07:12:58.362 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:58.362 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:58.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:12:58.412 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:58.412+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:58.432 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T07:12:58.446 INFO:teuthology.orchestra.run.smithi031.stdout:2627 trunc from 0xac050e2 to 0x78660fa 2024-09-13T07:12:58.460 INFO:teuthology.orchestra.run.smithi031.stdout:2629 read 0x22793e2 thru 0x227d3ad (0x3fcc bytes) 2024-09-13T07:12:58.471 INFO:teuthology.orchestra.run.smithi031.stdout:2630 write 0xceee3ab thru 0xcef7687 (0x92dd bytes) 2024-09-13T07:12:58.480 INFO:teuthology.orchestra.run.smithi031.stdout:2632 write 0x7009ead thru 0x7010ef1 (0x7045 bytes) 2024-09-13T07:12:58.503 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:58.503+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:58.504 INFO:teuthology.orchestra.run.smithi031.stdout:2633 trunc from 0xcef7688 to 0x118a74f 2024-09-13T07:12:58.549 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:58.549 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:58.556 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:58.556 INFO:teuthology.orchestra.run.smithi031.stdout:2635 punch from 0x475291 to 0x482488, (0xd1f7 bytes) 2024-09-13T07:12:58.557 INFO:teuthology.orchestra.run.smithi031.stdout:2636 punch from 0x614846 to 0x6194a8, (0x4c62 bytes) 2024-09-13T07:12:58.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:12:58.564 INFO:teuthology.orchestra.run.smithi031.stdout:2638 write 0x2a172c6 thru 0x2a203db (0x9116 bytes) 2024-09-13T07:12:58.572 INFO:teuthology.orchestra.run.smithi031.stdout:2639 punch from 0x23abea6 to 0x23ad2ce, (0x1428 bytes) 2024-09-13T07:12:58.578 INFO:teuthology.orchestra.run.smithi031.stdout:2640 trunc from 0x2a203dc to 0x74c2e18 2024-09-13T07:12:58.584 INFO:teuthology.orchestra.run.smithi031.stdout:2641 read 0x402b3c3 thru 0x403a8ca (0xf508 bytes) 2024-09-13T07:12:58.585 INFO:teuthology.orchestra.run.smithi031.stdout:2642 writesame 0x23df399 thru 0x23e4781 data_size 0x1(0x53e9 bytes) 2024-09-13T07:12:58.593 INFO:teuthology.orchestra.run.smithi031.stdout:2643 read 0x7316449 thru 0x7316a79 (0x631 bytes) 2024-09-13T07:12:58.604 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:58.603+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:58.607 INFO:teuthology.orchestra.run.smithi031.stdout:2644 writesame 0xdffa69e thru 0xe003f3e data_size 0x1(0x98a1 bytes) 2024-09-13T07:12:58.607 INFO:teuthology.orchestra.run.smithi031.stdout:2645 write 0x5084381 thru 0x5091237 (0xceb7 bytes) 2024-09-13T07:12:58.615 INFO:teuthology.orchestra.run.smithi031.stdout:2646 read 0x352b54 thru 0x3551b4 (0x2661 bytes) 2024-09-13T07:12:58.616 INFO:teuthology.orchestra.run.smithi031.stdout:2647 punch from 0x973b0d4 to 0x9741195, (0x60c1 bytes) 2024-09-13T07:12:58.616 INFO:teuthology.orchestra.run.smithi031.stdout:2648 read 0x8a0568f thru 0x8a0faae (0xa420 bytes) 2024-09-13T07:12:58.616 INFO:teuthology.orchestra.run.smithi031.stdout:2649 write 0xa47c76a thru 0xa48a628 (0xdebf bytes) 2024-09-13T07:12:58.624 INFO:teuthology.orchestra.run.smithi031.stdout:2650 write 0xb57e671 thru 0xb58cd6a (0xe6fa bytes) 2024-09-13T07:12:58.630 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:58.630 INFO:teuthology.orchestra.run.smithi031.stdout:2652 punch from 0x3edcbf7 to 0x3ee904e, (0xc457 bytes) 2024-09-13T07:12:58.633 INFO:teuthology.orchestra.run.smithi031.stdout:2653 trunc from 0xe003f3f to 0xaf336e3 2024-09-13T07:12:58.643 INFO:teuthology.orchestra.run.smithi031.stdout:2654 writesame 0x9a05c4d thru 0x9a09ae4 data_size 0x4(0x3e98 bytes) 2024-09-13T07:12:58.650 INFO:teuthology.orchestra.run.smithi031.stdout:2655 clone 16 order 20 su 131072 sc 4 2024-09-13T07:12:58.687 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:58.688 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:58.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:12:58.705 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:58.706+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:58.805 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:58.805+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:58.842 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:58.843 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:58.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:12:58.906 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:58.908+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:58.998 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:58.998 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:59.007 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:12:59.007 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:12:59.007+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:12:59.108 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:59.109+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:59.113 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:59.113 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:59.122 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:12:59.210 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:12:59.209+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:12:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:59.264 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:59.273 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:12:59.283 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:12:59.316 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:12:59.317+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:12:59.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:12:59.354 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone15 from 0xaf336e3 (overlap 0x118a74f) to 0x543231 2024-09-13T07:12:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:59.396 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:59.410 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:12:59.451 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:59.452+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:59.502 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:12:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:12:59.512 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:12:59.512+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:12:59.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:12:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:12:59.534 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:12:59.534 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:12:59.550 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:12:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:12:59.553 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:12:59.562 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:12:59.567 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:12:59.567 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:12:59.583 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:12:59.583 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:12:59.601 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:12:59.602 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:12:59.602 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:12:59.605 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:12:59.605 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:12:59.613 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:12:59.614+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:12:59.691 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:59.692 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:12:59.692 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:12:59.692 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:12:59.692 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:59.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:12:59.714 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:59.715+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:59.752 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T07:12:59.775 INFO:teuthology.orchestra.run.smithi031.stdout:2656 write 0xba41e9c thru 0xba4ca4f (0xabb4 bytes) 2024-09-13T07:12:59.778 INFO:teuthology.orchestra.run.smithi031.stdout:2657 read 0x4ebed15 thru 0x4ec6019 (0x7305 bytes) 2024-09-13T07:12:59.786 INFO:teuthology.orchestra.run.smithi031.stdout:2659 trunc from 0xba4ca50 to 0x4c86400 2024-09-13T07:12:59.800 INFO:teuthology.orchestra.run.smithi031.stdout:2661 writesame 0x33dff08 thru 0x33e1906 data_size 0xb(0x19ff bytes) 2024-09-13T07:12:59.804 INFO:teuthology.orchestra.run.smithi031.stdout:2662 read 0x26cd99b thru 0x26cdecd (0x533 bytes) 2024-09-13T07:12:59.804 INFO:teuthology.orchestra.run.smithi031.stdout:2663 punch from 0xb71b41 to 0xb72b55, (0x1014 bytes) 2024-09-13T07:12:59.804 INFO:teuthology.orchestra.run.smithi031.stdout:2664 read 0x2a2c6a1 thru 0x2a35f93 (0x98f3 bytes) 2024-09-13T07:12:59.805 INFO:teuthology.orchestra.run.smithi031.stdout:2665 punch from 0x2476538 to 0x247f048, (0x8b10 bytes) 2024-09-13T07:12:59.805 INFO:teuthology.orchestra.run.smithi031.stdout:2667 read 0x288dc5c thru 0x289430f (0x66b4 bytes) 2024-09-13T07:12:59.805 INFO:teuthology.orchestra.run.smithi031.stdout:2669 read 0xd29bde thru 0xd2bcfe (0x2121 bytes) 2024-09-13T07:12:59.815 INFO:teuthology.orchestra.run.smithi031.stdout:2670 writesame 0x82f6a1b thru 0x82f9e67 data_size 0x1(0x344d bytes) 2024-09-13T07:12:59.815 INFO:teuthology.orchestra.run.smithi031.stdout:2671 read 0x68d3b47 thru 0x68dc01f (0x84d9 bytes) 2024-09-13T07:12:59.815 INFO:teuthology.orchestra.run.smithi031.stdout:2673 punch from 0x72c4dd2 to 0x72cd0a6, (0x82d4 bytes) 2024-09-13T07:12:59.815 INFO:teuthology.orchestra.run.smithi031.stdout:2675 write 0x441ed26 thru 0x4422c91 (0x3f6c bytes) 2024-09-13T07:12:59.815 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:12:59.816+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:12:59.818 INFO:teuthology.orchestra.run.smithi031.stdout:2676 read 0x248f7d5 thru 0x2491496 (0x1cc2 bytes) 2024-09-13T07:12:59.818 INFO:teuthology.orchestra.run.smithi031.stdout:2677 write 0x1692aaf thru 0x16945b9 (0x1b0b bytes) 2024-09-13T07:12:59.826 INFO:teuthology.orchestra.run.smithi031.stdout:2679 trunc from 0x82f9e68 to 0x383eb19 2024-09-13T07:12:59.844 INFO:teuthology.orchestra.run.smithi031.stdout:2680 write 0x3b43337 thru 0x3b47fb8 (0x4c82 bytes) 2024-09-13T07:12:59.847 INFO:teuthology.orchestra.run.smithi031.stdout:2681 read 0x2da1b84 thru 0x2daf2d6 (0xd753 bytes) 2024-09-13T07:12:59.853 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:12:59.853 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:12:59.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:12:59.853 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:12:59.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1309 3.1d 3:bcefd1cd:::rbd_data.1173c53667c.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.389690599000001, 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014288309000000001, 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1309 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463746+0000", 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463745+0000", 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:12:59.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463751+0000", 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463761+0000", 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3589999999999997e-06 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.463767+0000", 2024-09-13T07:12:59.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4420000000000003e-06 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472731+0000", 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0089641479999999999 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.472858+0000", 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012669100000000001 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.473057+0000", 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019875900000000001 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477793+0000", 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0047361900000000004 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.477983+0000", 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019018299999999999 2024-09-13T07:12:59.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478014+0000", 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1062999999999997e-05 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:37.478034+0000", 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.978e-05 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21840 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027820+0000", 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8256167940000001, 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012957078, 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21840 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027820+0000", 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027820+0000", 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027819+0000", 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027821+0000", 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8989999999999999e-06 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027826+0000", 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2310000000000004e-06 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027831+0000", 2024-09-13T07:12:59.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1069999999999998e-06 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040661+0000", 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012830504 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040716+0000", 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5458000000000003e-05 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040777+0000", 2024-09-13T07:12:59.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0183999999999998e-05 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21841 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8256019290000003, 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013061882, 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21841 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027834+0000", 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027836+0000", 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8339999999999999e-06 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027841+0000", 2024-09-13T07:12:59.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4779999999999999e-06 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027846+0000", 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3279999999999999e-06 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040798+0000", 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.012952353999999999 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040841+0000", 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2707000000000003e-05 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040896+0000", 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5516999999999997e-05 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.868 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21842 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027850+0000", 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8255868130000001, 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013245323, 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21842 2024-09-13T07:12:59.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027850+0000", 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027850+0000", 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027849+0000", 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027851+0000", 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9640000000000002e-06 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027857+0000", 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4600000000000002e-06 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027861+0000", 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3590000000000001e-06 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.040924+0000", 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013063162999999999 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041004+0000", 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9406999999999994e-05 2024-09-13T07:12:59.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041095+0000", 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.1305999999999996e-05 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21843 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8255714940000001, 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013344300999999999, 2024-09-13T07:12:59.873 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21843 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027865+0000", 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027867+0000", 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9879999999999999e-06 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027872+0000", 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4430000000000002e-06 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027883+0000", 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1267000000000001e-05 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041120+0000", 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013236494999999999 2024-09-13T07:12:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041161+0000", 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1737999999999999e-05 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041209+0000", 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7709e-05 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21844 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8255484959999997, 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013515821000000001, 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21844 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027888+0000", 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027890+0000", 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.407e-06 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027896+0000", 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027900+0000", 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6489999999999999e-06 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041238+0000", 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013337521999999999 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041317+0000", 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9258000000000006e-05 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041404+0000", 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6489999999999994e-05 2024-09-13T07:12:59.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21845 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027905+0000", 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8255318440000003, 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013605556, 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.882 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21845 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027905+0000", 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027905+0000", 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027904+0000", 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027911+0000", 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4899999999999997e-06 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027917+0000", 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027921+0000", 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041422+0000", 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013500955 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041463+0000", 2024-09-13T07:12:59.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0828000000000003e-05 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041510+0000", 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7299999999999998e-05 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21846 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027925+0000", 2024-09-13T07:12:59.886 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8255115570000005, 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013726685000000001, 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21846 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027925+0000", 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027925+0000", 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027924+0000", 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027927+0000", 2024-09-13T07:12:59.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.215e-06 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027932+0000", 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027937+0000", 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4649999999999996e-06 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041532+0000", 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013595177999999999 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041589+0000", 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7564e-05 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041652+0000", 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2218000000000006e-05 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.890 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21847 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8254960640000002, 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013846261, 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21847 2024-09-13T07:12:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027940+0000", 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027942+0000", 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0820000000000001e-06 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027947+0000", 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4210000000000003e-06 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027952+0000", 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2230000000000001e-06 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041683+0000", 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013731577999999999 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041733+0000", 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9669999999999997e-05 2024-09-13T07:12:59.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041787+0000", 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3650000000000003e-05 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21848 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8254810560000001, 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013953712, 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.895 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21848 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027955+0000", 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027957+0000", 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0379999999999998e-06 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027963+0000", 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027967+0000", 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2560000000000004e-06 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041797+0000", 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013830488 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041846+0000", 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8865000000000001e-05 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041909+0000", 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2941000000000006e-05 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21849 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1327104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.899 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027971+0000", 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8254657759999997, 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014093507, 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21849 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027971+0000", 2024-09-13T07:12:59.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027971+0000", 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027970+0000", 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027972+0000", 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0080000000000001e-06 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027978+0000", 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3890000000000004e-06 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027982+0000", 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1230000000000002e-06 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041945+0000", 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.013963606999999999 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041996+0000", 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0949000000000003e-05 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042064+0000", 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.779e-05 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21850 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8254505349999999, 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014194567999999999, 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21850 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.904 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027986+0000", 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027988+0000", 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027993+0000", 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4759999999999997e-06 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.027997+0000", 2024-09-13T07:12:59.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1910000000000002e-06 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042088+0000", 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01409063 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042131+0000", 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3296e-05 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042180+0000", 2024-09-13T07:12:59.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9212000000000002e-05 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21851 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1507328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8254353830000003, 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014302794000000001, 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21851 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028001+0000", 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028003+0000", 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0099999999999998e-06 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028009+0000", 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6560000000000003e-06 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028014+0000", 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3050000000000003e-06 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042202+0000", 2024-09-13T07:12:59.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014188186 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042252+0000", 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0096999999999998e-05 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042304+0000", 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1841000000000001e-05 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21852 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8254189060000003, 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014397584, 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21852 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.913 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028017+0000", 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028019+0000", 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9860000000000001e-06 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028024+0000", 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2530000000000004e-06 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028029+0000", 2024-09-13T07:12:59.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2980000000000002e-06 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042324+0000", 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014295560000000001 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042367+0000", 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2534999999999998e-05 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042415+0000", 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8288999999999998e-05 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21853 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8254039649999996, 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014507977, 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21853 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028032+0000", 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028034+0000", 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0140000000000001e-06 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028039+0000", 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3870000000000003e-06 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028044+0000", 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6e-06 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042437+0000", 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014393056 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042483+0000", 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5859999999999998e-05 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042540+0000", 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7404999999999997e-05 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21854 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.921 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8253879770000001, 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014603028000000001, 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21854 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:59.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028048+0000", 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028050+0000", 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9350000000000001e-06 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028056+0000", 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028060+0000", 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4909999999999999e-06 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042561+0000", 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014501193000000001 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042602+0000", 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0831000000000003e-05 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042651+0000", 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9356e-05 2024-09-13T07:12:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21855 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 1998848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8253722699999999, 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014702378, 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.926 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21855 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028064+0000", 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028066+0000", 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.995e-06 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028071+0000", 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.383e-06 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028075+0000", 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3170000000000003e-06 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042672+0000", 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014596560999999999 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042717+0000", 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4994999999999999e-05 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042767+0000", 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9471000000000001e-05 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.930 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21856 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 2154496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028080+0000", 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8253566880000003, 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014797103000000001, 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21856 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028080+0000", 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028080+0000", 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028079+0000", 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028081+0000", 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.871e-06 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028086+0000", 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028091+0000", 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3579999999999996e-06 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042787+0000", 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014696407 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042829+0000", 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1499999999999999e-05 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042877+0000", 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8139000000000001e-05 2024-09-13T07:12:59.935 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21857 3.1c 3:393a1e1f:::rbd_data.117368475c62.0000000000000008:f [read 2179072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8253412080000002, 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014904802, 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.936 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21857 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028095+0000", 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028097+0000", 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0449999999999999e-06 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028102+0000", 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4020000000000001e-06 2024-09-13T07:12:59.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.028107+0000", 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2420000000000002e-06 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042898+0000", 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014791871 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.042942+0000", 2024-09-13T07:12:59.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3778e-05 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043000+0000", 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7819999999999999e-05 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:24084 3.c 3:3307f6ad:::rbd_data.11734c5d5131.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 2738537~1960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:56.663924+0000", 2024-09-13T07:12:59.940 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.189511907, 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015900592000000002, 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 24084 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.663924+0000", 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.663924+0000", 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.663922+0000", 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.670563+0000", 2024-09-13T07:12:59.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0066409449999999997 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.670578+0000", 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4161999999999999e-05 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.670601+0000", 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3643e-05 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:12:59.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.670635+0000", 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3887999999999998e-05 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.670731+0000", 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5623000000000002e-05 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.671712+0000", 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00098170799999999997 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.676907+0000", 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051949889999999997 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.679759+0000", 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028519029999999998 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.679796+0000", 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7045999999999997e-05 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:12:59.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:56.679825+0000", 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8765e-05 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:12:59.946 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:12:59.948 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:12:59.950 INFO:teuthology.orchestra.run.smithi031.stdout:2682 writesame 0xad5ad01 thru 0xad694dd data_size 0x1(0xe7dd bytes) 2024-09-13T07:12:59.950 INFO:teuthology.orchestra.run.smithi031.stdout:2683 write 0x755a495 thru 0x7565312 (0xae7e bytes) 2024-09-13T07:12:59.950 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2687 write 0xc7cfe5c thru 0xc7d4934 (0x4ad9 bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2688 read 0x9776c4d thru 0x9783a95 (0xce49 bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2691 trunc from 0xc7d4935 to 0xc608f42 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2692 read 0x84c4536 thru 0x84cb109 (0x6bd4 bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2693 punch from 0x5cc267d to 0x5cc93d1, (0x6d54 bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2696 punch from 0xb5c00e0 to 0xb5c9677, (0x9597 bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2697 trunc from 0xc608f42 to 0x770ba8d 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2698 punch from 0x496cb2 to 0x498aac, (0x1dfa bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2701 write 0x3eeb234 thru 0x3efa0fb (0xeec8 bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2703 punch from 0x3fb857f to 0x3fbd10c, (0x4b8d bytes) 2024-09-13T07:12:59.951 INFO:teuthology.orchestra.run.smithi031.stdout:2704 writesame 0x1cd488f thru 0x1cde5e2 data_size 0x2(0x9d54 bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2706 read 0x646704f thru 0x6469c4f (0x2c01 bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2707 write 0x73458c8 thru 0x7347d60 (0x2499 bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2708 punch from 0x5b0fb94 to 0x5b14312, (0x477e bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2710 punch from 0x1470faf to 0x147fb68, (0xebb9 bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2712 punch from 0x38712ae to 0x387e8c2, (0xd614 bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2717 write 0x45862f0 thru 0x458ad3b (0x4a4c bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2718 punch from 0x55ff998 to 0x5604725, (0x4d8d bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2719 read 0x5d15374 thru 0x5d1bb63 (0x67f0 bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2721 write 0x9a5e7ca thru 0x9a65247 (0x6a7e bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2722 write 0xe7949a1 thru 0xe798f9a (0x45fa bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2724 write 0x1fa4a66 thru 0x1faf1ce (0xa769 bytes) 2024-09-13T07:12:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:2726 writesame 0x51e7284 thru 0x51f6c22 data_size 0x1(0xf99f bytes) 2024-09-13T07:12:59.953 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:12:59.948+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:12:59.973 INFO:teuthology.orchestra.run.smithi031.stdout:2728 read 0x3811c5 thru 0x3852be (0x40fa bytes) 2024-09-13T07:12:59.974 INFO:teuthology.orchestra.run.smithi031.stdout:2732 write 0x29821da thru 0x2982729 (0x550 bytes) 2024-09-13T07:12:59.978 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:59.978 INFO:teuthology.orchestra.run.smithi031.stdout:2735 write 0xcab465f thru 0xcabb72e (0x70d0 bytes) 2024-09-13T07:12:59.982 INFO:teuthology.orchestra.run.smithi031.stdout:2736 read 0x871446f thru 0x8722657 (0xe1e9 bytes) 2024-09-13T07:12:59.982 INFO:teuthology.orchestra.run.smithi031.stdout:2737 read 0x15351be thru 0x15368c1 (0x1704 bytes) 2024-09-13T07:12:59.982 INFO:teuthology.orchestra.run.smithi031.stdout:2739 write 0x64e649c thru 0x64ea904 (0x4469 bytes) 2024-09-13T07:12:59.984 INFO:teuthology.orchestra.run.smithi031.stdout:2740 write 0xb0d935 thru 0xb1d206 (0xf8d2 bytes) 2024-09-13T07:12:59.994 INFO:teuthology.orchestra.run.smithi031.stdout:2741 read 0x1e291f3 thru 0x1e36519 (0xd327 bytes) 2024-09-13T07:12:59.995 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:12:59.995 INFO:teuthology.orchestra.run.smithi031.stdout:2743 write 0xbd71f7e thru 0xbd7c2e4 (0xa367 bytes) 2024-09-13T07:13:00.002 INFO:teuthology.orchestra.run.smithi031.stdout:2745 trunc from 0xe798f9b to 0xaa1f893 2024-09-13T07:13:00.012 INFO:teuthology.orchestra.run.smithi031.stdout:2748 read 0xa269a49 thru 0xa277f47 (0xe4ff bytes) 2024-09-13T07:13:00.012 INFO:teuthology.orchestra.run.smithi031.stdout:2751 writesame 0x623a21c thru 0x6243d58 data_size 0xd(0x9b3d bytes) 2024-09-13T07:13:00.017 INFO:teuthology.orchestra.run.smithi031.stdout:2752 write 0x4357b01 thru 0x4358802 (0xd02 bytes) 2024-09-13T07:13:00.018 INFO:teuthology.orchestra.run.smithi031.stdout:2753 write 0x6d0a2ea thru 0x6d14ee5 (0xabfc bytes) 2024-09-13T07:13:00.029 INFO:teuthology.orchestra.run.smithi031.stdout:2754 write 0xca6c28c thru 0xca6dfbf (0x1d34 bytes) 2024-09-13T07:13:00.032 INFO:teuthology.orchestra.run.smithi031.stdout:2755 write 0x1419688 thru 0x1426715 (0xd08e bytes) 2024-09-13T07:13:00.036 INFO:teuthology.orchestra.run.smithi031.stdout:2756 write 0x958013c thru 0x95898b1 (0x9776 bytes) 2024-09-13T07:13:00.038 INFO:teuthology.orchestra.run.smithi031.stdout:2758 read 0x6dd0d3a thru 0x6dd5022 (0x42e9 bytes) 2024-09-13T07:13:00.039 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:00.039 INFO:teuthology.orchestra.run.smithi031.stdout:2760 punch from 0x16fb808 to 0x16fd248, (0x1a40 bytes) 2024-09-13T07:13:00.048 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:00.049+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:00.051 INFO:teuthology.orchestra.run.smithi031.stdout:2763 trunc from 0xca6dfc0 to 0x16dff3c 2024-09-13T07:13:00.086 INFO:teuthology.orchestra.run.smithi031.stdout:2765 write 0xbc8ae3f thru 0xbc95f5a (0xb11c bytes) 2024-09-13T07:13:00.089 INFO:teuthology.orchestra.run.smithi031.stdout:2769 read 0x24bc604 thru 0x24bf071 (0x2a6e bytes) 2024-09-13T07:13:00.096 INFO:teuthology.orchestra.run.smithi031.stdout:2770 write 0xd870cdd thru 0xd87af3b (0xa25f bytes) 2024-09-13T07:13:00.099 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:00.099 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:00.099 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:00.099 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:00.106 INFO:teuthology.orchestra.run.smithi031.stdout:2772 write 0xebeff0f thru 0xebf95fe (0x96f0 bytes) 2024-09-13T07:13:00.109 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:00.109 INFO:teuthology.orchestra.run.smithi031.stdout:2773 write 0x8bd8bbc thru 0x8be74e7 (0xe92c bytes) 2024-09-13T07:13:00.113 INFO:teuthology.orchestra.run.smithi031.stdout:2774 write 0x736ee72 thru 0x73784f3 (0x9682 bytes) 2024-09-13T07:13:00.115 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:00.115 INFO:teuthology.orchestra.run.smithi031.stdout:2779 writesame 0xa793c09 thru 0xa796f05 data_size 0x13(0x32fd bytes) 2024-09-13T07:13:00.119 INFO:teuthology.orchestra.run.smithi031.stdout:2780 writesame 0x510fdad thru 0x5116a01 data_size 0x1(0x6c55 bytes) 2024-09-13T07:13:00.134 INFO:teuthology.orchestra.run.smithi031.stdout:2781 punch from 0x7e92dc0 to 0x7e99fb9, (0x71f9 bytes) 2024-09-13T07:13:00.134 INFO:teuthology.orchestra.run.smithi031.stdout:2782 write 0x6f31c9c thru 0x6f3ae15 (0x917a bytes) 2024-09-13T07:13:00.139 INFO:teuthology.orchestra.run.smithi031.stdout:2783 punch from 0x9d7c888 to 0x9d8b630, (0xeda8 bytes) 2024-09-13T07:13:00.139 INFO:teuthology.orchestra.run.smithi031.stdout:2785 punch from 0x3cf5f44 to 0x3cfaefe, (0x4fba bytes) 2024-09-13T07:13:00.139 INFO:teuthology.orchestra.run.smithi031.stdout:2786 read 0x34871dd thru 0x348d599 (0x63bd bytes) 2024-09-13T07:13:00.139 INFO:teuthology.orchestra.run.smithi031.stdout:2787 read 0x531b4d thru 0x534afb (0x2faf bytes) 2024-09-13T07:13:00.139 INFO:teuthology.orchestra.run.smithi031.stdout:2788 write 0x4dad65 thru 0x4e4e71 (0xa10d bytes) 2024-09-13T07:13:00.148 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:00.148+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:00.151 INFO:teuthology.orchestra.run.smithi031.stdout:2790 write 0x74cdc1e thru 0x74d81bc (0xa59f bytes) 2024-09-13T07:13:00.155 INFO:teuthology.orchestra.run.smithi031.stdout:2791 read 0x9c3c836 thru 0x9c491c0 (0xc98b bytes) 2024-09-13T07:13:00.155 INFO:teuthology.orchestra.run.smithi031.stdout:2793 write 0x936939b thru 0x9373a7d (0xa6e3 bytes) 2024-09-13T07:13:00.158 INFO:teuthology.orchestra.run.smithi031.stdout:2795 read 0x5316376 thru 0x53249ec (0xe677 bytes) 2024-09-13T07:13:00.158 INFO:teuthology.orchestra.run.smithi031.stdout:2796 write 0x5667dfb thru 0x566ab2f (0x2d35 bytes) 2024-09-13T07:13:00.162 INFO:teuthology.orchestra.run.smithi031.stdout:2797 writesame 0xa4d3de3 thru 0xa4def56 data_size 0x4(0xb174 bytes) 2024-09-13T07:13:00.169 INFO:teuthology.orchestra.run.smithi031.stdout:2798 punch from 0x59c363e to 0x59c3c5c, (0x61e bytes) 2024-09-13T07:13:00.169 INFO:teuthology.orchestra.run.smithi031.stdout:2799 writesame 0xaa0b454 thru 0xaa0fc83 data_size 0x30(0x4830 bytes) 2024-09-13T07:13:00.171 INFO:teuthology.orchestra.run.smithi031.stdout:2800 punch from 0x6b07989 to 0x6b17983, (0xfffa bytes) 2024-09-13T07:13:00.171 INFO:teuthology.orchestra.run.smithi031.stdout:2801 read 0xd2fb13 thru 0xd35864 (0x5d52 bytes) 2024-09-13T07:13:00.173 INFO:teuthology.orchestra.run.smithi031.stdout:2802 write 0x40b137e thru 0x40b9239 (0x7ebc bytes) 2024-09-13T07:13:00.175 INFO:teuthology.orchestra.run.smithi031.stdout:2803 read 0xbe0ffd8 thru 0xbe1ef07 (0xef30 bytes) 2024-09-13T07:13:00.176 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:00.187 INFO:teuthology.orchestra.run.smithi031.stdout:2805 trunc from 0xebf95ff to 0x4ee40d6 2024-09-13T07:13:00.210 INFO:teuthology.orchestra.run.smithi031.stdout:2806 write 0xa3ae8f6 thru 0xa3b1732 (0x2e3d bytes) 2024-09-13T07:13:00.213 INFO:teuthology.orchestra.run.smithi031.stdout:2807 read 0x2f00df9 thru 0x2f0849f (0x76a7 bytes) 2024-09-13T07:13:00.213 INFO:teuthology.orchestra.run.smithi031.stdout:2808 clone 17 order 24 su 4194304 sc 14 2024-09-13T07:13:00.249 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:00.250+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:00.255 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:00.255 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:00.255 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:00.255 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:00.255 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:00.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:00.350 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:00.350+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:00.365 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone16 2024-09-13T07:13:00.432 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.394977248, 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:13:00.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1768 3.a e25/19 ondisk, result = 0)", 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.380104162999999, 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010131984, 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052064+0000", 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:00.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052072+0000", 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9529999999999999e-06 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052090+0000", 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8263000000000001e-05 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052102+0000", 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2078e-05 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062047+0000", 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0099446580000000003 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062059+0000", 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2455e-05 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062197+0000", 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013813200000000001 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21398 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.4234422269999998, 2024-09-13T07:13:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088116559999999993, 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21398 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008729+0000", 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.798e-06 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008734+0000", 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.763e-06 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008738+0000", 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7999999999999998e-06 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011757+0000", 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030188580000000001 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011838+0000", 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1021999999999998e-05 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012183+0000", 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034440499999999999 2024-09-13T07:13:00.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013964+0000", 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001781071 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017492+0000", 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035283329999999998 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017520+0000", 2024-09-13T07:13:00.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7687000000000001e-05 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017539+0000", 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9126e-05 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21399 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.4234270569999996, 2024-09-13T07:13:00.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088642229999999992, 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21399 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008745+0000", 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:13:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008750+0000", 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008755+0000", 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6480000000000002e-06 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012580+0000", 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003825214 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012630+0000", 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9772999999999999e-05 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012831+0000", 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000201294 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013967+0000", 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001136176 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017566+0000", 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035988190000000001 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017590+0000", 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3490999999999999e-05 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017607+0000", 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21921 3.10 3:0f55d758:::rbd_data.1173f6773d36.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3952927659999999, 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065309330000000001, 2024-09-13T07:13:00.455 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21921 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036876+0000", 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038933+0000", 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020573900000000001 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038947+0000", 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4071e-05 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038955+0000", 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1839999999999999e-06 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038999+0000", 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3399000000000002e-05 2024-09-13T07:13:00.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039104+0000", 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104798 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039324+0000", 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00022014499999999999 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041941+0000", 2024-09-13T07:13:00.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026169420000000001 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043354+0000", 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001412792 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043385+0000", 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1767999999999998e-05 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043408+0000", 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2422999999999999e-05 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22131 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2542117389999996, 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076826359999999996, 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22131 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:00.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177957+0000", 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177960+0000", 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9079999999999999e-06 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177967+0000", 2024-09-13T07:13:00.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4219999999999997e-06 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177970+0000", 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9380000000000001e-06 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178788+0000", 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081823400000000004 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178835+0000", 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7348000000000001e-05 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179052+0000", 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021655000000000001 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.182995+0000", 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039430350000000001 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185592+0000", 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025971280000000002 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185618+0000", 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6390999999999999e-05 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185640+0000", 2024-09-13T07:13:00.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2144e-05 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22132 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2541830990000005, 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076953339999999999, 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.467 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22132 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177989+0000", 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6749999999999998e-06 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177995+0000", 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8839999999999999e-06 2024-09-13T07:13:00.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177999+0000", 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6650000000000002e-06 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179333+0000", 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001334029 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179381+0000", 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8115000000000003e-05 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179582+0000", 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020005199999999999 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183028+0000", 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003446185 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185646+0000", 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00261845 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185666+0000", 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0301000000000001e-05 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185682+0000", 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5333999999999999e-05 2024-09-13T07:13:00.472 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22133 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2541661829999997, 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007715502, 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.473 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22133 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178005+0000", 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.745e-06 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178008+0000", 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.354e-06 2024-09-13T07:13:00.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178011+0000", 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.847e-06 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179904+0000", 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001892828 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179977+0000", 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2959000000000004e-05 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180170+0000", 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019334699999999999 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183047+0000", 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028766690000000001 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185685+0000", 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002638315 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185704+0000", 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8929000000000001e-05 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185719+0000", 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4772e-05 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.478 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22134 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2541560159999996, 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077416129999999996, 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22134 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.479 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178013+0000", 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178015+0000", 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.449e-06 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178018+0000", 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0970000000000002e-06 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178021+0000", 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9790000000000002e-06 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180446+0000", 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002425207 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180494+0000", 2024-09-13T07:13:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7870000000000001e-05 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180680+0000", 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018602899999999999 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183074+0000", 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023943660000000002 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185722+0000", 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026480010000000001 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185741+0000", 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8328000000000001e-05 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185755+0000", 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4552999999999999e-05 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.484 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22135 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2541467739999996, 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007772101, 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22135 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:00.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178024+0000", 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3909999999999999e-06 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178027+0000", 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178029+0000", 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4679999999999999e-06 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180968+0000", 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029389149999999998 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181015+0000", 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7178999999999997e-05 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181200+0000", 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018424700000000001 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183091+0000", 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001891413 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185758+0000", 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026670819999999999 2024-09-13T07:13:00.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185778+0000", 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9656e-05 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185795+0000", 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7020999999999998e-05 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22384 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0378082299999996, 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078826639999999993, 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22384 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:00.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394363+0000", 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.511e-06 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394368+0000", 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4589999999999997e-06 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394371+0000", 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397357+0000", 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029860659999999999 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397406+0000", 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8982e-05 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397659+0000", 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025325 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.401969+0000", 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043103020000000002 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402177+0000", 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020735 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402215+0000", 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8532999999999999e-05 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402244+0000", 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8756999999999999e-05 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.496 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22563 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8901080260000001, 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065088769999999997, 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22563 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542061+0000", 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542063+0000", 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4810000000000001e-06 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542066+0000", 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.954e-06 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542068+0000", 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6280000000000001e-06 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546580+0000", 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045117259999999998 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546648+0000", 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7398999999999996e-05 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546893+0000", 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024524099999999999 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548346+0000", 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001453153 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548486+0000", 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013988499999999999 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548532+0000", 2024-09-13T07:13:00.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6675000000000003e-05 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548570+0000", 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7938000000000002e-05 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22607 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:00.502 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.856543072, 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067359300000000002, 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22607 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575629+0000", 2024-09-13T07:13:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.943e-06 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575635+0000", 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0249999999999999e-06 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575640+0000", 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0900000000000004e-06 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.578914+0000", 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032737140000000001 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579018+0000", 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000103928 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579262+0000", 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000243988 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.581291+0000", 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020291419999999998 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582314+0000", 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010232660000000001 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582343+0000", 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8844e-05 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582362+0000", 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9409e-05 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22608 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8565253229999996, 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097116479999999998, 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22608 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575646+0000", 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575655+0000", 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3359999999999995e-06 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575660+0000", 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579853+0000", 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004193123 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579939+0000", 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6316999999999994e-05 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580288+0000", 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034927800000000001 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582212+0000", 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019236679999999999 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585304+0000", 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030922509999999999 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585337+0000", 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585356+0000", 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8899e-05 2024-09-13T07:13:00.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22609 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8564982670000001, 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097598709999999998, 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22609 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575674+0000", 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0850000000000001e-06 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575685+0000", 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195e-05 2024-09-13T07:13:00.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575690+0000", 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3589999999999999e-06 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580716+0000", 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050255289999999999 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580764+0000", 2024-09-13T07:13:00.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8402000000000002e-05 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580983+0000", 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021850999999999999 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585136+0000", 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041530070000000002 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585385+0000", 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024898500000000002 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585412+0000", 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7240999999999999e-05 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585431+0000", 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8995e-05 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22684 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8022018509999995, 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069291209999999999, 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22684 2024-09-13T07:13:00.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629967+0000", 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629969+0000", 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.468e-06 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629972+0000", 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0520000000000002e-06 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629975+0000", 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634438+0000", 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044633980000000004 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634515+0000", 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7020000000000005e-05 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634768+0000", 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025338500000000001 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636038+0000", 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012701800000000001 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636808+0000", 2024-09-13T07:13:00.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076938199999999999 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636854+0000", 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6149000000000001e-05 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636897+0000", 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2796999999999997e-05 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.525 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22685 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8021929669999999, 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072232559999999999, 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22685 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.526 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629976+0000", 2024-09-13T07:13:00.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629978+0000", 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.42e-06 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629981+0000", 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7630000000000001e-06 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629983+0000", 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.424e-06 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635281+0000", 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052976969999999996 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635373+0000", 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2596000000000006e-05 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635650+0000", 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027690599999999999 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636553+0000", 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000902646 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637134+0000", 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058095900000000003 2024-09-13T07:13:00.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637169+0000", 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4793999999999999e-05 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637200+0000", 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1254999999999998e-05 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22783 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.717170265, 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062902130000000002, 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22783 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.714999+0000", 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715002+0000", 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8430000000000001e-06 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715009+0000", 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5379999999999997e-06 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715014+0000", 2024-09-13T07:13:00.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.719299+0000", 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042849819999999997 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.719381+0000", 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2181999999999993e-05 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.719740+0000", 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00035867400000000002 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721214+0000", 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001474508 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721223+0000", 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5010000000000004e-06 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721267+0000", 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4359e-05 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721289+0000", 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2585000000000001e-05 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22784 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.537 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.7171514850000005, 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072328160000000004, 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22784 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:00.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715020+0000", 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715026+0000", 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9190000000000004e-06 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715031+0000", 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5889999999999996e-06 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720306+0000", 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005275475 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720370+0000", 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3802999999999999e-05 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720620+0000", 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025008499999999999 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721297+0000", 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00067708199999999997 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722163+0000", 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00086599500000000005 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722217+0000", 2024-09-13T07:13:00.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3393999999999997e-05 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722251+0000", 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4223000000000001e-05 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22798 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.7086563379999999, 2024-09-13T07:13:00.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0064185800000000001, 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22798 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723513+0000", 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:00.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723516+0000", 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.208e-06 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723524+0000", 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3720000000000005e-06 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.723531+0000", 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3339999999999998e-06 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.727445+0000", 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039143729999999996 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.727528+0000", 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3009000000000004e-05 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.727889+0000", 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00036129800000000002 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.728636+0000", 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00074646599999999999 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.729858+0000", 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001222239 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.729911+0000", 2024-09-13T07:13:00.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3397999999999999e-05 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.729932+0000", 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0387e-05 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:00.549 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:00.551 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:00.552+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:00.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:00.615 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T07:13:00.651 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:00.652+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:00.676 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:00.684 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:00.684 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:00.684 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:00.684 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:00.685 INFO:teuthology.orchestra.run.smithi031.stdout:2810 trunc from 0xa3b1733 to 0x2c6b257 2024-09-13T07:13:00.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:00.729 INFO:teuthology.orchestra.run.smithi031.stdout:2811 write 0x7fee420 thru 0x7ff11d5 (0x2db6 bytes) 2024-09-13T07:13:00.731 INFO:teuthology.orchestra.run.smithi031.stdout:2812 read 0x5193f14 thru 0x519b76a (0x7857 bytes) 2024-09-13T07:13:00.731 INFO:teuthology.orchestra.run.smithi031.stdout:2813 read 0x31d2069 thru 0x31e1dea (0xfd82 bytes) 2024-09-13T07:13:00.731 INFO:teuthology.orchestra.run.smithi031.stdout:2816 read 0x12b2ff3 thru 0x12bce1e (0x9e2c bytes) 2024-09-13T07:13:00.731 INFO:teuthology.orchestra.run.smithi031.stdout:2817 read 0xc81a4c thru 0xc8ca19 (0xafce bytes) 2024-09-13T07:13:00.732 INFO:teuthology.orchestra.run.smithi031.stdout:2818 writesame 0x3f2fd5f thru 0x3f3f534 data_size 0x1(0xf7d6 bytes) 2024-09-13T07:13:00.752 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:00.752+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:00.755 INFO:teuthology.orchestra.run.smithi031.stdout:2819 punch from 0x795a8d0 to 0x795e10e, (0x383e bytes) 2024-09-13T07:13:00.756 INFO:teuthology.orchestra.run.smithi031.stdout:2820 write 0x2a7bf38 thru 0x2a889c2 (0xca8b bytes) 2024-09-13T07:13:00.757 INFO:teuthology.orchestra.run.smithi031.stdout:2821 read 0x1c6ba5d thru 0x1c6f2d1 (0x3875 bytes) 2024-09-13T07:13:00.758 INFO:teuthology.orchestra.run.smithi031.stdout:2822 read 0x66525f7 thru 0x6655302 (0x2d0c bytes) 2024-09-13T07:13:00.758 INFO:teuthology.orchestra.run.smithi031.stdout:2823 read 0x3d14f10 thru 0x3d1a8b1 (0x59a2 bytes) 2024-09-13T07:13:00.766 INFO:teuthology.orchestra.run.smithi031.stdout:2824 trunc from 0x7ff11d6 to 0xe91d251 2024-09-13T07:13:00.770 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:00.770 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:00.770 INFO:teuthology.orchestra.run.smithi031.stdout:2828 read 0x773c682 thru 0x77427f7 (0x6176 bytes) 2024-09-13T07:13:00.770 INFO:teuthology.orchestra.run.smithi031.stdout:2831 read 0xa08c52e thru 0xa092aa6 (0x6579 bytes) 2024-09-13T07:13:00.771 INFO:teuthology.orchestra.run.smithi031.stdout:2832 write 0x4132d4d thru 0x4140557 (0xd80b bytes) 2024-09-13T07:13:00.772 INFO:teuthology.orchestra.run.smithi031.stdout:2833 clone 18 order 20 su 4096 sc 6 2024-09-13T07:13:00.811 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:00.811 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:00.811 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:00.812 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:00.812 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:00.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:00.862 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:00.863+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:00.954 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:00.955+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:00.969 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:00.969 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:00.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:00.969 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:00.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32783 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1921024~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431805+0000", 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53752860400000002, 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023923267000000002, 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32783 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:00.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431805+0000", 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431805+0000", 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431804+0000", 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:00.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431806+0000", 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1310000000000001e-06 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431808+0000", 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7949999999999999e-06 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431811+0000", 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455576+0000", 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023764838 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455655+0000", 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8646999999999994e-05 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455728+0000", 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3421000000000001e-05 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32785 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1937408~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53752037399999997, 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024093753999999998, 2024-09-13T07:13:00.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32785 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:00.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:00.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:00.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:00.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:00.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:00.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:00.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431814+0000", 2024-09-13T07:13:00.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1680000000000001e-06 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431818+0000", 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9790000000000004e-06 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431820+0000", 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.48e-06 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455754+0000", 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023934033 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455831+0000", 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6688999999999997e-05 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455907+0000", 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5609000000000001e-05 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32787 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1953792~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:00.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53751090000000001, 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024269968999999999, 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32787 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:00.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:00.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431824+0000", 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5090000000000001e-06 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431827+0000", 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3859999999999999e-06 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431831+0000", 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9310000000000001e-06 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455934+0000", 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024102704999999999 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456024+0000", 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0750999999999999e-05 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456092+0000", 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7986999999999998e-05 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32790 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1974272~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53748316200000001, 2024-09-13T07:13:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024485522999999999, 2024-09-13T07:13:00.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:00.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:00.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:00.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:00.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32790 2024-09-13T07:13:00.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.989 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:00.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:00.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:00.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:00.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:00.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:00.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:00.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:00.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:00.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431851+0000", 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1930000000000001e-06 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431854+0000", 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9409999999999999e-06 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:00.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431858+0000", 2024-09-13T07:13:00.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-13T07:13:00.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:00.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456119+0000", 2024-09-13T07:13:00.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024260993000000002 2024-09-13T07:13:00.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:00.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456198+0000", 2024-09-13T07:13:00.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8625e-05 2024-09-13T07:13:00.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:00.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456336+0000", 2024-09-13T07:13:00.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013795699999999999 2024-09-13T07:13:00.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:00.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:00.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32792 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1990656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:00.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:00.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53745827000000002, 2024-09-13T07:13:00.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024634488999999999, 2024-09-13T07:13:00.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:00.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:00.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32792 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:00.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431876+0000", 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2249999999999999e-06 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:00.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431879+0000", 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-06 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431882+0000", 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6120000000000001e-06 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456363+0000", 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024481266000000002 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456441+0000", 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7888999999999999e-05 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456510+0000", 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32794 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2007040~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53743487400000001, 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024779235, 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32794 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431899+0000", 2024-09-13T07:13:01.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.254e-06 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431902+0000", 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431905+0000", 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456537+0000", 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024632001000000001 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456613+0000", 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5838000000000001e-05 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456678+0000", 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4684000000000006e-05 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32796 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2023424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431917+0000", 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53741656500000001, 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024950045000000001, 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32796 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431917+0000", 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431917+0000", 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431916+0000", 2024-09-13T07:13:01.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431918+0000", 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.232e-06 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431921+0000", 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8779999999999998e-06 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431924+0000", 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1250000000000001e-06 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456702+0000", 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024778535000000001 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456782+0000", 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9942000000000001e-05 2024-09-13T07:13:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456867+0000", 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4560000000000004e-05 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32806 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1912832~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53727485200000002, 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024993043999999999, 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32806 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432060+0000", 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.277e-06 2024-09-13T07:13:01.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432066+0000", 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432072+0000", 2024-09-13T07:13:01.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9390000000000002e-06 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456907+0000", 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024834730999999999 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456983+0000", 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5972999999999999e-05 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457051+0000", 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8644000000000002e-05 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32808 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1929216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432120+0000", 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53721354899999996, 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025101938000000001, 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32808 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432120+0000", 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432120+0000", 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432119+0000", 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432122+0000", 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.931e-06 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432129+0000", 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7279999999999998e-06 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432134+0000", 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457079+0000", 2024-09-13T07:13:01.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024944814999999999 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457153+0000", 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4485000000000003e-05 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457222+0000", 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8268000000000004e-05 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32810 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1945600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53717917999999998, 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025251326000000001, 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32810 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:01.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432156+0000", 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0549999999999998e-06 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432161+0000", 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432166+0000", 2024-09-13T07:13:01.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7190000000000001e-06 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457249+0000", 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025082598000000001 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457322+0000", 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3701000000000005e-05 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457405+0000", 2024-09-13T07:13:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2966000000000005e-05 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32812 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1961984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432193+0000", 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53714074099999998, 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025603027, 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32812 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432193+0000", 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432193+0000", 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432192+0000", 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432194+0000", 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2199999999999999e-06 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432201+0000", 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:13:01.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432205+0000", 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457434+0000", 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025228424999999999 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457684+0000", 2024-09-13T07:13:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025016699999999998 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457796+0000", 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111632 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32829 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1978368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432439+0000", 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53689468900000004, 2024-09-13T07:13:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025503896000000002, 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32829 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432439+0000", 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432439+0000", 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432438+0000", 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432440+0000", 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432446+0000", 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9379999999999997e-06 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432451+0000", 2024-09-13T07:13:01.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4989999999999996e-06 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457822+0000", 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025371044999999998 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457905+0000", 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3311000000000002e-05 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457942+0000", 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7353e-05 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32831 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1994752~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432455+0000", 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53687841199999997, 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025646097, 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32831 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432455+0000", 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432455+0000", 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432454+0000", 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432461+0000", 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.37e-06 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432468+0000", 2024-09-13T07:13:01.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4240000000000004e-06 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432473+0000", 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9350000000000002e-06 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457973+0000", 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025499786 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458032+0000", 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209000000000002e-05 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458101+0000", 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8824000000000004e-05 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32833 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2011136~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.037 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.536856004, 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025797789000000002, 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32833 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432479+0000", 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432486+0000", 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432490+0000", 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.476e-06 2024-09-13T07:13:01.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458129+0000", 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025638882000000002 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458209+0000", 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9345999999999996e-05 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458275+0000", 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000006e-05 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32835 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2027520~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.433060+0000", 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.53627374800000005, 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025397137, 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32835 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433060+0000", 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433060+0000", 2024-09-13T07:13:01.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432496+0000", 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994364 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433063+0000", 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056719300000000002 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433073+0000", 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9469999999999998e-06 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433080+0000", 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6669999999999999e-06 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458315+0000", 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025235218 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458391+0000", 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6252000000000001e-05 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458457+0000", 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5432999999999995e-05 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33286 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1843200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439893+0000", 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.52944045299999998, 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024047123, 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33286 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439893+0000", 2024-09-13T07:13:01.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439893+0000", 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439892+0000", 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439895+0000", 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6220000000000001e-06 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439902+0000", 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1670000000000002e-06 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439907+0000", 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463795+0000", 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023887512999999999 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463867+0000", 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1967999999999999e-05 2024-09-13T07:13:01.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463940+0000", 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3081e-05 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33287 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1859584~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439912+0000", 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.52942153300000006, 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024187856000000001, 2024-09-13T07:13:01.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33287 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439912+0000", 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439912+0000", 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439911+0000", 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439914+0000", 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3599999999999999e-06 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439920+0000", 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2469999999999997e-06 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439925+0000", 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6090000000000003e-06 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463965+0000", 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024039957000000001 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464037+0000", 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2469e-05 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464100+0000", 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2575000000000003e-05 2024-09-13T07:13:01.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33288 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1875968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439929+0000", 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.529404706, 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024327224000000001, 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33288 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439929+0000", 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439929+0000", 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439928+0000", 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439930+0000", 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439936+0000", 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5809999999999996e-06 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439940+0000", 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2209999999999999e-06 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464124+0000", 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024183818999999999 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464193+0000", 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8891000000000002e-05 2024-09-13T07:13:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464256+0000", 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2910999999999994e-05 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33289 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1892352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.52938913600000004, 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024469930000000001, 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33289 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439946+0000", 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9989999999999998e-06 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439951+0000", 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.524e-06 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439955+0000", 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.98e-06 2024-09-13T07:13:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464282+0000", 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024327147 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464354+0000", 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2050000000000003e-05 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464414+0000", 2024-09-13T07:13:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9632000000000001e-05 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33317 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1896448~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.52605197000000004, 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024472794999999999, 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33317 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443283+0000", 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443290+0000", 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443296+0000", 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3410000000000002e-06 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.467136+0000", 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023840222000000001 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.467637+0000", 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050127199999999998 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.467754+0000", 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001171 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.070 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:01.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:01.072 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:01.072+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:01.171 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:01.173+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:01.218 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:01.219 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:01.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:01.272 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:01.273+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:01.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:13:01.340 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:01.340 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:01.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:01.350 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone17 from 0xe91d251 (overlap 0x2c6b257) to 0x1f17cd6 2024-09-13T07:13:01.394 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:01.395+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:01.432 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T07:13:01.475 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:01.475 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:01.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:01.475 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:01.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4257.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.890677625000002, 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01398422, 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4257", 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/806497074", 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584056+0000", 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584054+0000", 2024-09-13T07:13:01.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584061+0000", 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1929999999999998e-06 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584069+0000", 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8240000000000006e-06 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584077+0000", 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584133+0000", 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5896000000000002e-05 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.584145+0000", 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2293000000000001e-05 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593792+0000", 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096463009999999995 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.593914+0000", 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121994 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.594023+0000", 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109152 2024-09-13T07:13:01.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597204+0000", 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003181021 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597968+0000", 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076444399999999997 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.597998+0000", 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0051000000000001e-05 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:20.598040+0000", 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1548999999999998e-05 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7113 3.19 3:98be421e:::rbd_data.11736f46b2f8.0000000000000006:9 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.781416918, 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088087640000000002, 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7113 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693316+0000", 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693318+0000", 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6759999999999999e-06 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693324+0000", 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8900000000000004e-06 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693329+0000", 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9139999999999999e-06 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.701974+0000", 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086443170000000003 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702050+0000", 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6386999999999999e-05 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702125+0000", 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5057999999999999e-05 2024-09-13T07:13:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7114 3.19 3:9ddc4a4c:::rbd_data.11736f46b2f8.0000000000000007:9 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.781399218000001, 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089814739999999997, 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7114 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693334+0000", 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693336+0000", 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2340000000000001e-06 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693344+0000", 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3310000000000004e-06 2024-09-13T07:13:01.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693349+0000", 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1549999999999998e-06 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702155+0000", 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088061089999999995 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702237+0000", 2024-09-13T07:13:01.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1180999999999994e-05 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702315+0000", 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8845999999999997e-05 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7115 3.19 3:9ddc4a4c:::rbd_data.11736f46b2f8.0000000000000007:9 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:13:01.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.781378798, 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091461110000000002, 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7115 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693354+0000", 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693357+0000", 2024-09-13T07:13:01.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693365+0000", 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2300000000000008e-06 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.693370+0000", 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702342+0000", 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089718839999999994 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702420+0000", 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7986999999999997e-05 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:48.702500+0000", 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0829999999999997e-05 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7725 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.045898838999999, 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086548819999999992, 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7725 2024-09-13T07:13:01.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428834+0000", 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428838+0000", 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3359999999999997e-06 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428847+0000", 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5289999999999997e-06 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428852+0000", 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0130000000000001e-06 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.432776+0000", 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039239380000000001 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.432867+0000", 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0299999999999999e-05 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433090+0000", 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022336900000000001 2024-09-13T07:13:01.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434316+0000", 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001225547 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437408+0000", 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030926180000000001 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437455+0000", 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437489+0000", 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4663000000000003e-05 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7726 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.428964+0000", 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.045769045, 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093293319999999992, 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7726 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428964+0000", 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428964+0000", 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428963+0000", 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428969+0000", 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428980+0000", 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1435e-05 2024-09-13T07:13:01.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.428987+0000", 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5710000000000003e-06 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433485+0000", 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044981860000000004 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433558+0000", 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3308999999999999e-05 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.433958+0000", 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000399923 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438195+0000", 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042367280000000004 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438204+0000", 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6349999999999995e-06 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438252+0000", 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8922999999999999e-05 2024-09-13T07:13:01.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438293+0000", 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0989999999999999e-05 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7727 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429188+0000", 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.0455451, 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092464539999999994, 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7727 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429188+0000", 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429188+0000", 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429187+0000", 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429207+0000", 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.995e-05 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429227+0000", 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0097000000000001e-05 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429236+0000", 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0920000000000004e-06 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434341+0000", 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051047599999999999 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434417+0000", 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5958999999999996e-05 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434619+0000", 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020198699999999999 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438306+0000", 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036867649999999998 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438352+0000", 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879000000000003e-05 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438397+0000", 2024-09-13T07:13:01.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5068000000000003e-05 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438434+0000", 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7045000000000002e-05 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.516 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7728 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429262+0000", 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.045471137, 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093129170000000004, 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7728 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.517 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429262+0000", 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429262+0000", 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429261+0000", 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429268+0000", 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.392e-06 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429277+0000", 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8049999999999996e-06 2024-09-13T07:13:01.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429282+0000", 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.434983+0000", 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057009419999999996 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435037+0000", 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3696000000000002e-05 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435238+0000", 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000200674 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438311+0000", 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030733829999999998 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438495+0000", 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184079 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438540+0000", 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4428000000000002e-05 2024-09-13T07:13:01.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.438575+0000", 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5339000000000002e-05 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7729 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429393+0000", 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.045340387, 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011020118000000001, 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7729 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429393+0000", 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429393+0000", 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429392+0000", 2024-09-13T07:13:01.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429396+0000", 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3080000000000001e-06 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429403+0000", 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5570000000000001e-06 2024-09-13T07:13:01.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429407+0000", 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9720000000000003e-06 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435587+0000", 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061800049999999997 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.435678+0000", 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1380999999999998e-05 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436091+0000", 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041248700000000001 2024-09-13T07:13:01.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439596+0000", 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035056060000000001 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440367+0000", 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077025499999999996 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440393+0000", 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6670999999999999e-05 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440413+0000", 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9538000000000002e-05 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7730 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429524+0000", 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.04520928, 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010984203, 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7730 2024-09-13T07:13:01.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429524+0000", 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429524+0000", 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429523+0000", 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429528+0000", 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.04e-06 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429541+0000", 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867000000000001e-05 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429548+0000", 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0299999999999996e-06 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436671+0000", 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071234640000000004 2024-09-13T07:13:01.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436756+0000", 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4546e-05 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437081+0000", 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032500999999999998 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439914+0000", 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028333020000000002 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440450+0000", 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053587299999999997 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440475+0000", 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4763e-05 2024-09-13T07:13:01.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440508+0000", 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3229999999999999e-05 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7731 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.429641+0000", 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.045092165, 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010952033, 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7731 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429641+0000", 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429641+0000", 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429640+0000", 2024-09-13T07:13:01.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429646+0000", 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4339999999999997e-06 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429660+0000", 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.429668+0000", 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7520000000000002e-06 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437534+0000", 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078658840000000001 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437586+0000", 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1758999999999997e-05 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437783+0000", 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019671700000000001 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439920+0000", 2024-09-13T07:13:01.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021370719999999998 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440549+0000", 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00062917400000000003 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440575+0000", 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6261000000000001e-05 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.440593+0000", 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7657000000000001e-05 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7966 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665139+0000", 2024-09-13T07:13:01.544 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.809594246, 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086921749999999999, 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7966 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665139+0000", 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665139+0000", 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665138+0000", 2024-09-13T07:13:01.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665144+0000", 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0140000000000004e-06 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665156+0000", 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1579999999999999e-05 2024-09-13T07:13:01.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665166+0000", 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0081999999999999e-05 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.670352+0000", 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051865929999999998 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.670426+0000", 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3530999999999998e-05 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.670928+0000", 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050263100000000002 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672306+0000", 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001378017 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673782+0000", 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001475924 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673812+0000", 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9261999999999999e-05 2024-09-13T07:13:01.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673831+0000", 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9500999999999998e-05 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7967 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665216+0000", 2024-09-13T07:13:01.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.809517011000001, 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087581970000000006, 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7967 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665216+0000", 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665216+0000", 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665215+0000", 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665220+0000", 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9830000000000004e-06 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665230+0000", 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7890000000000006e-06 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665236+0000", 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.671394+0000", 2024-09-13T07:13:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061572270000000004 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.671458+0000", 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4614000000000002e-05 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.671747+0000", 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028857699999999999 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673842+0000", 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020953999999999999 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673888+0000", 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5401000000000001e-05 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673932+0000", 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4373999999999999e-05 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673974+0000", 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2412e-05 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7968 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665245+0000", 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.809488219, 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096363979999999991, 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7968 2024-09-13T07:13:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665245+0000", 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665245+0000", 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665244+0000", 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665248+0000", 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6859999999999999e-06 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665256+0000", 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7470000000000005e-06 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665261+0000", 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-06 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672332+0000", 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070711209999999997 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672422+0000", 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9598999999999994e-05 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.672724+0000", 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030285300000000001 2024-09-13T07:13:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674028+0000", 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013034279999999999 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674800+0000", 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077240800000000004 2024-09-13T07:13:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674847+0000", 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6371e-05 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674881+0000", 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4656000000000002e-05 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7969 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665344+0000", 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.809388909000001, 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098855350000000008, 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7969 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665344+0000", 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665344+0000", 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665343+0000", 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665349+0000", 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6799999999999998e-06 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665359+0000", 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665366+0000", 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673121+0000", 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077545390000000004 2024-09-13T07:13:01.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673203+0000", 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2095000000000006e-05 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.673512+0000", 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030892000000000002 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674951+0000", 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00143925 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675166+0000", 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000214976 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675194+0000", 2024-09-13T07:13:01.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8390000000000001e-05 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675230+0000", 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5318999999999998e-05 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:7970 3.0 3:02f6b5f6:::rbd_object_map.117344605995:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.665529+0000", 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.809204363999999, 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098965349999999997, 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7970 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665529+0000", 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665529+0000", 2024-09-13T07:13:01.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665528+0000", 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665533+0000", 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9359999999999998e-06 2024-09-13T07:13:01.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665547+0000", 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4453e-05 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.665555+0000", 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8099999999999998e-06 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674060+0000", 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085048309999999992 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674136+0000", 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5694999999999999e-05 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.674440+0000", 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030381700000000001 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675235+0000", 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079571899999999996 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675339+0000", 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010414800000000001 2024-09-13T07:13:01.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675389+0000", 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9529e-05 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.675425+0000", 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6396999999999997e-05 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:8347 3.19 3:9f4f66e3:::rbd_data.117344605995.0000000000000168:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.070765+0000", 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.403968604999999, 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013952958999999999, 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 8347 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070765+0000", 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070765+0000", 2024-09-13T07:13:01.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070764+0000", 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070769+0000", 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8720000000000001e-06 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070778+0000", 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1900000000000001e-06 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070783+0000", 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070820+0000", 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7472000000000003e-05 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.070910+0000", 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9815000000000007e-05 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.073366+0000", 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002455718 2024-09-13T07:13:01.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084601+0000", 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011234675 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084647+0000", 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084678+0000", 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1315000000000001e-05 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.084717+0000", 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9643e-05 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:30887 3.4 3:2447e438:::rbd_data.11738e0d4a46.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:59.905175+0000", 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.569558413, 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010187897, 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30887 2024-09-13T07:13:01.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905175+0000", 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905175+0000", 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905174+0000", 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905180+0000", 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7710000000000003e-06 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905190+0000", 2024-09-13T07:13:01.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0886e-05 2024-09-13T07:13:01.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905196+0000", 2024-09-13T07:13:01.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1309999999999997e-06 2024-09-13T07:13:01.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905244+0000", 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8016999999999998e-05 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.905390+0000", 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146539 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.907133+0000", 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017425920000000001 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.915263+0000", 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081301529999999993 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.915308+0000", 2024-09-13T07:13:01.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5253999999999997e-05 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.915341+0000", 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2515000000000003e-05 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.915363+0000", 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2005e-05 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:30919 3.0 3:0314ddcc:::rbd_data.11738e0d4a46.0000000000000053:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:59.950607+0000", 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5241257990000001, 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024181601, 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:01.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30919 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950607+0000", 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950607+0000", 2024-09-13T07:13:01.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950606+0000", 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950615+0000", 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2250000000000006e-06 2024-09-13T07:13:01.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950632+0000", 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6807999999999999e-05 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950638+0000", 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7279999999999998e-06 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950681+0000", 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2128000000000001e-05 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.950783+0000", 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010223899999999999 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.955261+0000", 2024-09-13T07:13:01.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044780549999999999 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.974638+0000", 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019376793999999999 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.974685+0000", 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7487000000000001e-05 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.974747+0000", 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1649999999999994e-05 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.974789+0000", 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2203000000000002e-05 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4467.0:30919 3.0 e42/19 ondisk, result = 0)", 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:59.964428+0000", 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5103049180000001, 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010369679999999999, 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.964428+0000", 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.964428+0000", 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.964427+0000", 2024-09-13T07:13:01.604 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:01.475+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:01.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.964432+0000", 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9540000000000003e-06 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.964447+0000", 2024-09-13T07:13:01.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5153999999999999e-05 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.964456+0000", 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4989999999999994e-06 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.974668+0000", 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010212429 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.974678+0000", 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7710000000000001e-06 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:59.974798+0000", 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119837 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:01.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:01.608 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:01.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:01.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:01.609 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:01.611+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:01.627 INFO:teuthology.orchestra.run.smithi031.stdout:2834 writesame 0xd08b37f thru 0xd09892b data_size 0x1(0xd5ad bytes) 2024-09-13T07:13:01.710 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:01.711+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:01.746 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:01.746 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:01.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:01.763 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:01.763 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:01.780 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:01.781 INFO:teuthology.orchestra.run.smithi031.stdout:2835 read 0x842622d thru 0x8426f59 (0xd2d bytes) 2024-09-13T07:13:01.782 INFO:teuthology.orchestra.run.smithi031.stdout:2836 read 0xa4f39e0 thru 0xa4f7828 (0x3e49 bytes) 2024-09-13T07:13:01.782 INFO:teuthology.orchestra.run.smithi031.stdout:2838 write 0xbd5a89e thru 0xbd66810 (0xbf73 bytes) 2024-09-13T07:13:01.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:01.808 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:01.811 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:01.812+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:01.831 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:01.831 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:01.853 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:01.853 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:01.884 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:01.885 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:01.907 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:13:01.910 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:13:01.911 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:13:01.912 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:01.913+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:01.926 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:01.926 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:01.937 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:01.937 INFO:teuthology.orchestra.run.smithi031.stdout:2840 write 0x9082672 thru 0x9091e97 (0xf826 bytes) 2024-09-13T07:13:01.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:02.013 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:02.014+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:02.113 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:02.114+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:02.127 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:02.128 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:02.132 INFO:teuthology.orchestra.run.smithi031.stdout:2841 punch from 0x758c5a7 to 0x7599e42, (0xd89b bytes) 2024-09-13T07:13:02.132 INFO:teuthology.orchestra.run.smithi031.stdout:2842 read 0x4d8ff3 thru 0x4e5b2c (0xcb3a bytes) 2024-09-13T07:13:02.135 INFO:teuthology.orchestra.run.smithi031.stdout:2843 trunc from 0xe91d251 to 0xe0fe1bc 2024-09-13T07:13:02.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:02.149 INFO:teuthology.orchestra.run.smithi031.stdout:2845 trunc from 0xe0fe1bc to 0xe9f5604 2024-09-13T07:13:02.153 INFO:teuthology.orchestra.run.smithi031.stdout:2846 write 0xb4e43d1 thru 0xb4ebc70 (0x78a0 bytes) 2024-09-13T07:13:02.193 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-13T07:13:02.193 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:02.193 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:13:02.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:13:02.238 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:02.239+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:02.293 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:02.293 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:02.298 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:13:02.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:02.317 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:02.317+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:02.334 INFO:teuthology.orchestra.run.smithi031.stdout:2848 writesame 0x665b70c thru 0x6669d35 data_size 0x11(0xe62a bytes) 2024-09-13T07:13:02.417 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:02.419+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:02.461 INFO:teuthology.orchestra.run.smithi031.stdout:2849 punch from 0x712449c to 0x712f46a, (0xafce bytes) 2024-09-13T07:13:02.463 INFO:teuthology.orchestra.run.smithi031.stdout:2851 trunc from 0xe9f5604 to 0xca0feb3 2024-09-13T07:13:02.480 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:02.480 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:02.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:02.518 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:02.518+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:02.619 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:02.619+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 45, 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "f667c631-bb9d-47bf-97c6-7217d8893244", 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T07:12:08.346514+0000", 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T07:13:02.291594+0000", 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T07:12:46.205660+0000", 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T07:12:10.626165+0000", 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-13T07:13:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T07:13:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:19.268005+0000", 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 17, 2024-09-13T07:13:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.9900000095367432, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.9900000095367432, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T07:13:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:20.513150+0000", 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T07:13:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 7.690000057220459, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 7.690000057220459, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.12999999523162842, 2024-09-13T07:13:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1, 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1, 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:28.796583+0000", 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:13:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T07:13:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "44", 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 21, 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 44, 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:13:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.9900000095367432, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.9900000095367432, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c29fb342-d45b-48c1-9694-dd01e2a01325", 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 31, 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T07:13:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T07:13:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/1709347595", 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/1709347595", 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/1709347595", 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/1709347595", 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4b979b5c-a78b-461d-b795-f6f5c25cf652", 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.855316162109375, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 31, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T07:13:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3300827638", 2024-09-13T07:13:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3300827638", 2024-09-13T07:13:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3300827638", 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3300827638", 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "11259f54-1134-495f-9281-d5132aff2780", 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3977233096", 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3977233096", 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3977233096", 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3977233096", 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "a11be2be-3803-4a23-bbf8-40ea110ccf41", 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 31, 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T07:13:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T07:13:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/3290545828", 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/3290545828", 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/3290545828", 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/3290545828", 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2b6941d9-29f5-4836-a61c-47ff27a7172e", 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 31, 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6800", 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6801", 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6802", 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6803", 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6806", 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6807", 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6804", 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6805", 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6801/3543694628", 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6803/3543694628", 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6807/3543694628", 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6805/3543694628", 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "17e3a02d-28e3-4126-a8f2-c7783e524d62", 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 31, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6808", 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6810", 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6812", 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6814", 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6820", 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6821", 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6816", 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6818", 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6810/951246042", 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6814/951246042", 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6821/951246042", 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6818/951246042", 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:13:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ddf29f57-7733-42fa-94ed-015a529fb689", 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6824", 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6825", 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6826", 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6827", 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6830", 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6831", 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6828", 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6829", 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6825/1941674146", 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6827/1941674146", 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6831/1941674146", 2024-09-13T07:13:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6829/1941674146", 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d91e2eed-3bfb-4eb0-ac30-18770c106742", 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 0, 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 31, 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6809", 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6811", 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6813", 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6815", 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6822", 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6823", 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6817", 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6819", 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 322162340 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6811/322162340", 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6815/322162340", 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6823/322162340", 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6819/322162340", 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "exists" 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.805494+0000", 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.817444+0000", 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.712083+0000", 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.739958+0000", 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.720767+0000", 2024-09-13T07:13:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.715944+0000", 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:13:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.788853+0000", 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.809325+0000", 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 31 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T07:13:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T07:13:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T07:13:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:13:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:13:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T07:13:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:02.701 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:02.703 INFO:tasks.thrashosds.thrasher:j is {'epoch': 45, 'fsid': 'f667c631-bb9d-47bf-97c6-7217d8893244', 'created': '2024-09-13T07:12:08.346514+0000', 'modified': '2024-09-13T07:13:02.291594+0000', 'last_up_change': '2024-09-13T07:12:46.205660+0000', 'last_in_change': '2024-09-13T07:12:10.626165+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:12:19.268005+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:12:20.513150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:12:28.796583+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'c29fb342-d45b-48c1-9694-dd01e2a01325', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 1709347595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 1709347595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 1709347595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 1709347595}]}, 'public_addr': '172.21.15.31:6809/1709347595', 'cluster_addr': '172.21.15.31:6811/1709347595', 'heartbeat_back_addr': '172.21.15.31:6815/1709347595', 'heartbeat_front_addr': '172.21.15.31:6813/1709347595', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4b979b5c-a78b-461d-b795-f6f5c25cf652', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.855316162109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3300827638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3300827638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3300827638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3300827638}]}, 'public_addr': '172.21.15.31:6825/3300827638', 'cluster_addr': '172.21.15.31:6827/3300827638', 'heartbeat_back_addr': '172.21.15.31:6831/3300827638', 'heartbeat_front_addr': '172.21.15.31:6829/3300827638', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '11259f54-1134-495f-9281-d5132aff2780', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3977233096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3977233096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3977233096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3977233096}]}, 'public_addr': '172.21.15.31:6801/3977233096', 'cluster_addr': '172.21.15.31:6803/3977233096', 'heartbeat_back_addr': '172.21.15.31:6807/3977233096', 'heartbeat_front_addr': '172.21.15.31:6805/3977233096', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a11be2be-3803-4a23-bbf8-40ea110ccf41', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 3290545828}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 3290545828}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 3290545828}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 3290545828}]}, 'public_addr': '172.21.15.31:6817/3290545828', 'cluster_addr': '172.21.15.31:6819/3290545828', 'heartbeat_back_addr': '172.21.15.31:6823/3290545828', 'heartbeat_front_addr': '172.21.15.31:6821/3290545828', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2b6941d9-29f5-4836-a61c-47ff27a7172e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 3543694628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 3543694628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 3543694628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 3543694628}]}, 'public_addr': '172.21.15.184:6801/3543694628', 'cluster_addr': '172.21.15.184:6803/3543694628', 'heartbeat_back_addr': '172.21.15.184:6807/3543694628', 'heartbeat_front_addr': '172.21.15.184:6805/3543694628', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '17e3a02d-28e3-4126-a8f2-c7783e524d62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6810', 'nonce': 951246042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6814', 'nonce': 951246042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 951246042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6818', 'nonce': 951246042}]}, 'public_addr': '172.21.15.184:6810/951246042', 'cluster_addr': '172.21.15.184:6814/951246042', 'heartbeat_back_addr': '172.21.15.184:6821/951246042', 'heartbeat_front_addr': '172.21.15.184:6818/951246042', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ddf29f57-7733-42fa-94ed-015a529fb689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 1941674146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 1941674146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 1941674146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 1941674146}]}, 'public_addr': '172.21.15.184:6825/1941674146', 'cluster_addr': '172.21.15.184:6827/1941674146', 'heartbeat_back_addr': '172.21.15.184:6831/1941674146', 'heartbeat_front_addr': '172.21.15.184:6829/1941674146', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd91e2eed-3bfb-4eb0-ac30-18770c106742', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6809', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 322162340}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6813', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 322162340}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 322162340}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6817', 'nonce': 322162340}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 322162340}]}, 'public_addr': '172.21.15.184:6811/322162340', 'cluster_addr': '172.21.15.184:6815/322162340', 'heartbeat_back_addr': '172.21.15.184:6823/322162340', 'heartbeat_front_addr': '172.21.15.184:6819/322162340', 'state': ['exists']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.805494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.817444+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.712083+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.739958+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.720767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.715944+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.788853+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.809325+0000', 'dead_epoch': 31}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:13:02.703 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-13T07:13:02.703 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:02.704 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:02.704 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:02.704 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:02.705 INFO:teuthology.orchestra.run.smithi031.stdout:2853 trunc from 0xca0feb3 to 0x5b67de3 2024-09-13T07:13:02.720 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:02.720+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:02.820 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:02.820+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:02.828 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:02.828 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:02.840 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:02.891 INFO:teuthology.orchestra.run.smithi031.stdout:2855 trunc from 0x5b67de3 to 0xadf6dbf 2024-09-13T07:13:02.900 INFO:teuthology.orchestra.run.smithi031.stdout:2856 trunc from 0xadf6dbf to 0xe3cae50 2024-09-13T07:13:02.905 INFO:teuthology.orchestra.run.smithi031.stdout:2857 writesame 0xb1ae0af thru 0xb1ae8f0 data_size 0x7(0x842 bytes) 2024-09-13T07:13:02.921 INFO:teuthology.orchestra.run.smithi031.stdout:2858 trunc from 0xe3cae50 to 0x2d58bd9 2024-09-13T07:13:02.921 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:02.922+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:02.996 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:02.996 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.005 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:03.049 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:03.049+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:03.086 INFO:teuthology.orchestra.run.smithi031.stdout:2859 write 0x33f793b thru 0x33f83b2 (0xa78 bytes) 2024-09-13T07:13:03.102 INFO:teuthology.orchestra.run.smithi031.stdout:2861 write 0xa735051 thru 0xa744af1 (0xfaa1 bytes) 2024-09-13T07:13:03.110 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:03.110 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.115 INFO:teuthology.orchestra.run.smithi031.stdout:2862 write 0xc533c4b thru 0xc540525 (0xc8db bytes) 2024-09-13T07:13:03.120 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:03.122 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:03.123+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:03.122 INFO:teuthology.orchestra.run.smithi031.stdout:2863 read 0x5b161e6 thru 0x5b22433 (0xc24e bytes) 2024-09-13T07:13:03.128 INFO:teuthology.orchestra.run.smithi031.stdout:2865 write 0xd39fb30 thru 0xd3a861a (0x8aeb bytes) 2024-09-13T07:13:03.147 INFO:teuthology.orchestra.run.smithi031.stdout:2866 trunc from 0xd3a861b to 0x424c6dc 2024-09-13T07:13:03.174 INFO:teuthology.orchestra.run.smithi031.stdout:2869 write 0xe6ca46d thru 0xe6d60d6 (0xbc6a bytes) 2024-09-13T07:13:03.179 INFO:teuthology.orchestra.run.smithi031.stdout:2870 write 0xdea6c6e thru 0xdeb4645 (0xd9d8 bytes) 2024-09-13T07:13:03.187 INFO:teuthology.orchestra.run.smithi031.stdout:2873 write 0x48181c6 thru 0x4824d5e (0xcb99 bytes) 2024-09-13T07:13:03.194 INFO:teuthology.orchestra.run.smithi031.stdout:2875 write 0x321be9e thru 0x321e340 (0x24a3 bytes) 2024-09-13T07:13:03.197 INFO:teuthology.orchestra.run.smithi031.stdout:2876 writesame 0x8f0ce93 thru 0x8f15154 data_size 0x15(0x82c2 bytes) 2024-09-13T07:13:03.208 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:03.208 INFO:teuthology.orchestra.run.smithi031.stdout:2878 read 0x343a6c0 thru 0x3441949 (0x728a bytes) 2024-09-13T07:13:03.208 INFO:teuthology.orchestra.run.smithi031.stdout:2879 writesame 0x8c3a254 thru 0x8c3a847 data_size 0x2(0x5f4 bytes) 2024-09-13T07:13:03.222 INFO:teuthology.orchestra.run.smithi031.stdout:2880 trunc from 0xe6d60d7 to 0x3066326 2024-09-13T07:13:03.223 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:03.223+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:03.246 INFO:teuthology.orchestra.run.smithi031.stdout:2881 write 0x79b91fc thru 0x79c2992 (0x9797 bytes) 2024-09-13T07:13:03.251 INFO:teuthology.orchestra.run.smithi031.stdout:2882 write 0x23c7afa thru 0x23c9abd (0x1fc4 bytes) 2024-09-13T07:13:03.269 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:03.269 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.278 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:03.324 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:03.325+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:03.358 INFO:teuthology.orchestra.run.smithi031.stdout:2883 read 0x4298879 thru 0x42997d0 (0xf58 bytes) 2024-09-13T07:13:03.358 INFO:teuthology.orchestra.run.smithi031.stdout:2884 clone 19 order 23 su 131072 sc 7 2024-09-13T07:13:03.420 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:03.420 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.425 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:03.426+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:03.428 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:03.526 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:03.527+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:03.530 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:03.530 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.539 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:03.626 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:03.627+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:03.640 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:03.640 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.649 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:03.727 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:03.726+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:03.750 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:03.751 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.759 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:03.843 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:03.843+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:03.901 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:03.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:03.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:03.955 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:03.956+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:04.028 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:04.028 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:04.030 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:04.030+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:04.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:04.043 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:04.043 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:04.060 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:04.060 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:04.077 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:04.078 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:04.095 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:04.111 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:04.112 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:04.128 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:04.130 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:04.131+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:04.145 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:04.145 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:04.145 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:13:04.148 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:13:04.149 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:13:04.152 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:04.153 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:04.153 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:04.153 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:04.153 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:04.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:04.232 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:04.233+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:04.311 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32783 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1921024~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431805+0000", 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8793504049999998, 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023923267000000002, 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32783 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431805+0000", 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431805+0000", 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431804+0000", 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431806+0000", 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1310000000000001e-06 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431808+0000", 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7949999999999999e-06 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431811+0000", 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455576+0000", 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023764838 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455655+0000", 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8646999999999994e-05 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455728+0000", 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3421000000000001e-05 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32785 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1937408~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8793421750000001, 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024093753999999998, 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32785 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431813+0000", 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431814+0000", 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1680000000000001e-06 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431818+0000", 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9790000000000004e-06 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431820+0000", 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.48e-06 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455754+0000", 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023934033 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455831+0000", 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6688999999999997e-05 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455907+0000", 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5609000000000001e-05 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32787 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1953792~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.879332701, 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024269968999999999, 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32787 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431822+0000", 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431824+0000", 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5090000000000001e-06 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431827+0000", 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3859999999999999e-06 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431831+0000", 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9310000000000001e-06 2024-09-13T07:13:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.455934+0000", 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024102704999999999 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456024+0000", 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0750999999999999e-05 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456092+0000", 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7986999999999998e-05 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32790 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1974272~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8793049630000001, 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024485522999999999, 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32790 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431850+0000", 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431851+0000", 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1930000000000001e-06 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431854+0000", 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9409999999999999e-06 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431858+0000", 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456119+0000", 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024260993000000002 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456198+0000", 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8625e-05 2024-09-13T07:13:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456336+0000", 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013795699999999999 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32792 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1990656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8792800710000002, 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024634488999999999, 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32792 2024-09-13T07:13:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431875+0000", 2024-09-13T07:13:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431876+0000", 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2249999999999999e-06 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431879+0000", 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-06 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431882+0000", 2024-09-13T07:13:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6120000000000001e-06 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456363+0000", 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024481266000000002 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456441+0000", 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7888999999999999e-05 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456510+0000", 2024-09-13T07:13:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32794 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2007040~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8792566750000002, 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024779235, 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32794 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431898+0000", 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431899+0000", 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.254e-06 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431902+0000", 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431905+0000", 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456537+0000", 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024632001000000001 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456613+0000", 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5838000000000001e-05 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456678+0000", 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4684000000000006e-05 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32796 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2023424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.431917+0000", 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.879238366, 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024950045000000001, 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32796 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431917+0000", 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431917+0000", 2024-09-13T07:13:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431916+0000", 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431918+0000", 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.232e-06 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431921+0000", 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8779999999999998e-06 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.431924+0000", 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1250000000000001e-06 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456702+0000", 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024778535000000001 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456782+0000", 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9942000000000001e-05 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456867+0000", 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4560000000000004e-05 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32806 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1912832~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.879096653, 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024993043999999999, 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32806 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432058+0000", 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432060+0000", 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.277e-06 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432066+0000", 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432072+0000", 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9390000000000002e-06 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456907+0000", 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024834730999999999 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.456983+0000", 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5972999999999999e-05 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457051+0000", 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8644000000000002e-05 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32808 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1929216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432120+0000", 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8790353500000001, 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025101938000000001, 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32808 2024-09-13T07:13:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432120+0000", 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432120+0000", 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432119+0000", 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432122+0000", 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.931e-06 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432129+0000", 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7279999999999998e-06 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432134+0000", 2024-09-13T07:13:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457079+0000", 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024944814999999999 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457153+0000", 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4485000000000003e-05 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457222+0000", 2024-09-13T07:13:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8268000000000004e-05 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32810 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1945600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8790009809999999, 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025251326000000001, 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32810 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432154+0000", 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432156+0000", 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0549999999999998e-06 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432161+0000", 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432166+0000", 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7190000000000001e-06 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457249+0000", 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025082598000000001 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457322+0000", 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3701000000000005e-05 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457405+0000", 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2966000000000005e-05 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32812 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1961984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432193+0000", 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.878962542, 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025603027, 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32812 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432193+0000", 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432193+0000", 2024-09-13T07:13:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432192+0000", 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432194+0000", 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2199999999999999e-06 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432201+0000", 2024-09-13T07:13:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432205+0000", 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457434+0000", 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025228424999999999 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457684+0000", 2024-09-13T07:13:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025016699999999998 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457796+0000", 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111632 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32829 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1978368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432439+0000", 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.87871649, 2024-09-13T07:13:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025503896000000002, 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32829 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432439+0000", 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432439+0000", 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432438+0000", 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432440+0000", 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:13:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432446+0000", 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9379999999999997e-06 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432451+0000", 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4989999999999996e-06 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457822+0000", 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025371044999999998 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457905+0000", 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3311000000000002e-05 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457942+0000", 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7353e-05 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32831 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1994752~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432455+0000", 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8787002130000001, 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025646097, 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32831 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432455+0000", 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432455+0000", 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432454+0000", 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432461+0000", 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.37e-06 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432468+0000", 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4240000000000004e-06 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432473+0000", 2024-09-13T07:13:04.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9350000000000002e-06 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.457973+0000", 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025499786 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458032+0000", 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209000000000002e-05 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458101+0000", 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8824000000000004e-05 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32833 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2011136~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8786778050000001, 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025797789000000002, 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32833 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432477+0000", 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432479+0000", 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432486+0000", 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432490+0000", 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.476e-06 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458129+0000", 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025638882000000002 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458209+0000", 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9345999999999996e-05 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458275+0000", 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000006e-05 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:32835 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 2027520~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.433060+0000", 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8780955490000002, 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025397137, 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32835 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433060+0000", 2024-09-13T07:13:04.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433060+0000", 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.432496+0000", 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994364 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433063+0000", 2024-09-13T07:13:04.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056719300000000002 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433073+0000", 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9469999999999998e-06 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.433080+0000", 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6669999999999999e-06 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458315+0000", 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025235218 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458391+0000", 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6252000000000001e-05 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.458457+0000", 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5432999999999995e-05 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33286 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1843200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439893+0000", 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8712622539999999, 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024047123, 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33286 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439893+0000", 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439893+0000", 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439892+0000", 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439895+0000", 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6220000000000001e-06 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439902+0000", 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1670000000000002e-06 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439907+0000", 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463795+0000", 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023887512999999999 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463867+0000", 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1967999999999999e-05 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463940+0000", 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3081e-05 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33287 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1859584~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439912+0000", 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8712433339999999, 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024187856000000001, 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33287 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439912+0000", 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439912+0000", 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439911+0000", 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439914+0000", 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3599999999999999e-06 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439920+0000", 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2469999999999997e-06 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439925+0000", 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6090000000000003e-06 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.463965+0000", 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024039957000000001 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464037+0000", 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2469e-05 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464100+0000", 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2575000000000003e-05 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33288 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1875968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439929+0000", 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8712265070000003, 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024327224000000001, 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33288 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439929+0000", 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439929+0000", 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439928+0000", 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439930+0000", 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439936+0000", 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5809999999999996e-06 2024-09-13T07:13:04.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439940+0000", 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2209999999999999e-06 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464124+0000", 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024183818999999999 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464193+0000", 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8891000000000002e-05 2024-09-13T07:13:04.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464256+0000", 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2910999999999994e-05 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33289 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1892352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8712109369999999, 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024469930000000001, 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33289 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:04.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439944+0000", 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439946+0000", 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9989999999999998e-06 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439951+0000", 2024-09-13T07:13:04.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.524e-06 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.439955+0000", 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.98e-06 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464282+0000", 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024327147 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464354+0000", 2024-09-13T07:13:04.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2050000000000003e-05 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.464414+0000", 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9632000000000001e-05 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:33317 3.17 3:ec0dd397:::rbd_data.117393521298.0000000000000006:12 [read 1896448~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8678737710000002, 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024472794999999999, 2024-09-13T07:13:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33317 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443281+0000", 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443283+0000", 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443290+0000", 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.443296+0000", 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3410000000000002e-06 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.467136+0000", 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023840222000000001 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.467637+0000", 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050127199999999998 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.467754+0000", 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001171 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.398 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:04.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:13:04.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:04.401 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone18 from 0x79c2993 (overlap 0x2d58bd9) to 0xd9a9f4 2024-09-13T07:13:04.401 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:04.401+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:04.501 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:04.501+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:04.601 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:04.601+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:04.628 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:04.628 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:04.628 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:04.629 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:04.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:04.702 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:04.702+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:04.768 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:04.769 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:04.769 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:04.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:04.802 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:04.802+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:04.903 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:04.903+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:04.934 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49523 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2699264~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.49186866899999998, 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056285390999999997, 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49523 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442063+0000", 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9379999999999999e-06 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442068+0000", 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442073+0000", 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497684+0000", 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055610930000000003 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497759+0000", 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5199999999999998e-05 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.498346+0000", 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058719100000000004 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49609 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 4075520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47380433799999999, 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055911012000000003, 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49609 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460127+0000", 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1430000000000001e-06 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460133+0000", 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5729999999999998e-06 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460138+0000", 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515783+0000", 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055645873999999998 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515861+0000", 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7979000000000007e-05 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.516036+0000", 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017504900000000001 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49618 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 999424~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47128993000000002, 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055816179, 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49618 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462639+0000", 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462642+0000", 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462647+0000", 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462652+0000", 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518291+0000", 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055639051000000002 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518368+0000", 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6913e-05 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518456+0000", 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8219000000000004e-05 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49619 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47127350499999998, 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055989537999999998, 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49619 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462659+0000", 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462665+0000", 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462669+0000", 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518483+0000", 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055813437 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518559+0000", 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5981000000000003e-05 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518646+0000", 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7175999999999993e-05 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49620 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1048576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.4712556, 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056152684000000001, 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49620 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462676+0000", 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462683+0000", 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462688+0000", 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518672+0000", 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055983988999999998 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518747+0000", 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4633000000000002e-05 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518827+0000", 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000002e-05 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49621 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1073152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47123685199999998, 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056323023999999999, 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49621 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462695+0000", 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462701+0000", 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462706+0000", 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6530000000000002e-06 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518854+0000", 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056147858000000002 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518934+0000", 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0813e-05 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519016+0000", 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1434999999999995e-05 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49622 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1097728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47121956599999998, 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056490088000000001, 2024-09-13T07:13:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49622 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462712+0000", 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462718+0000", 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462722+0000", 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519042+0000", 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056320041000000001 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519120+0000", 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7607000000000004e-05 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519200+0000", 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166999999999993e-05 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49640 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1318912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46741486300000001, 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056480083, 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49640 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466514+0000", 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466517+0000", 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466524+0000", 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466529+0000", 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520731+0000", 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054202469000000003 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520808+0000", 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6854000000000007e-05 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522995+0000", 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00218681 2024-09-13T07:13:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49653 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1490944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46729365099999998, 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055956026999999998, 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49653 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466638+0000", 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466643+0000", 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466648+0000", 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-13T07:13:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522424+0000", 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055775681000000001 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522499+0000", 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5828000000000007e-05 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522592+0000", 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2781000000000005e-05 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49674 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1662976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46588026199999999, 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056005177000000003, 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49674 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468052+0000", 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.497e-06 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468058+0000", 2024-09-13T07:13:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2430000000000002e-06 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468063+0000", 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523872+0000", 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055808706999999999 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523960+0000", 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8919999999999996e-05 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.524055+0000", 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4222999999999996e-05 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49695 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.463594601, 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056046466000000003, 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49695 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470338+0000", 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470345+0000", 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4100000000000002e-06 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470351+0000", 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525756+0000", 2024-09-13T07:13:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055404954999999999 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525819+0000", 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2779000000000003e-05 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526382+0000", 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056291800000000003 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49696 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46357390300000001, 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056222355000000002, 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49696 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470355+0000", 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470358+0000", 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470364+0000", 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470369+0000", 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526411+0000", 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056041476999999999 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526489+0000", 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7787000000000006e-05 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526578+0000", 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9606999999999998e-05 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49704 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2056192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.461948052, 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055974318000000002, 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49704 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471981+0000", 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471984+0000", 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471990+0000", 2024-09-13T07:13:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7229999999999998e-06 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471995+0000", 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527788+0000", 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055793091000000003 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527861+0000", 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3038000000000001e-05 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527956+0000", 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4536000000000004e-05 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49706 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2080768~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46193024500000002, 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056135461999999997, 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49706 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472006+0000", 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472012+0000", 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472017+0000", 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527980+0000", 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055963185999999998 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528050+0000", 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979000000000002e-05 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528135+0000", 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4594999999999999e-05 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49708 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2105344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.461908084, 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056284837999999997, 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49708 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472021+0000", 2024-09-13T07:13:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472024+0000", 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.261e-06 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472029+0000", 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472034+0000", 2024-09-13T07:13:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0069999999999997e-06 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528161+0000", 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056126285999999997 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528232+0000", 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1000000000000005e-05 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528306+0000", 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4876000000000006e-05 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49711 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2129920~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46189057, 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444877999999997, 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49711 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472041+0000", 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472047+0000", 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-09-13T07:13:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472052+0000", 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528331+0000", 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056278504 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528399+0000", 2024-09-13T07:13:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7941000000000006e-05 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528484+0000", 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5249999999999999e-05 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49713 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2154496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46171925699999999, 2024-09-13T07:13:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444079000000001, 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49713 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472214+0000", 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472228+0000", 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3964999999999999e-05 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472236+0000", 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4109999999999999e-06 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528509+0000", 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056273532000000001 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528581+0000", 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1338e-05 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528655+0000", 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3858999999999999e-05 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49728 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2326528~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.45984535599999998, 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055971704999999997, 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49728 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474087+0000", 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474093+0000", 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474098+0000", 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529880+0000", 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055782102 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529967+0000", 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6853000000000003e-05 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.530056+0000", 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9430000000000003e-05 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49767 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2695168~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.45575568, 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062028049000000002, 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49767 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478176+0000", 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478182+0000", 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478187+0000", 2024-09-13T07:13:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532704+0000", 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054517200000000002 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532779+0000", 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5244e-05 2024-09-13T07:13:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.540202+0000", 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074227429999999999 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:50078 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.4419535, 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055871918999999999, 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50078 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491979+0000", 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491986+0000", 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491991+0000", 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544160+0000", 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052168953999999997 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544241+0000", 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1213999999999999e-05 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.547848+0000", 2024-09-13T07:13:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003606663 2024-09-13T07:13:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.022 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:05.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:05.024 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T07:13:05.024 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:05.024+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:05.053 INFO:teuthology.orchestra.run.smithi031.stdout:2885 write 0x1933b34 thru 0x1938626 (0x4af3 bytes) 2024-09-13T07:13:05.124 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:05.124+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:05.139 INFO:teuthology.orchestra.run.smithi031.stdout:2886 read 0x5d220a4 thru 0x5d26f93 (0x4ef0 bytes) 2024-09-13T07:13:05.149 INFO:teuthology.orchestra.run.smithi031.stdout:2887 writesame 0xba66518 thru 0xba696fc data_size 0x1(0x31e5 bytes) 2024-09-13T07:13:05.149 INFO:teuthology.orchestra.run.smithi031.stdout:2889 punch from 0xba262d4 to 0xba353ad, (0xf0d9 bytes) 2024-09-13T07:13:05.149 INFO:teuthology.orchestra.run.smithi031.stdout:2890 punch from 0x7e34886 to 0x7e34d6a, (0x4e4 bytes) 2024-09-13T07:13:05.149 INFO:teuthology.orchestra.run.smithi031.stdout:2891 punch from 0x37b02cc to 0x37bc5d3, (0xc307 bytes) 2024-09-13T07:13:05.149 INFO:teuthology.orchestra.run.smithi031.stdout:2892 write 0x9bd2776 thru 0x9bdb86e (0x90f9 bytes) 2024-09-13T07:13:05.152 INFO:teuthology.orchestra.run.smithi031.stdout:2893 punch from 0x5b428d2 to 0x5b44e90, (0x25be bytes) 2024-09-13T07:13:05.153 INFO:teuthology.orchestra.run.smithi031.stdout:2894 writesame 0x590297f thru 0x590f3ae data_size 0x28(0xca30 bytes) 2024-09-13T07:13:05.163 INFO:teuthology.orchestra.run.smithi031.stdout:2898 writesame 0xc7e9d11 thru 0xc7f9487 data_size 0x3(0xf777 bytes) 2024-09-13T07:13:05.163 INFO:teuthology.orchestra.run.smithi031.stdout:2900 write 0x3414835 thru 0x3414b33 (0x2ff bytes) 2024-09-13T07:13:05.219 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:05.219 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:05.225 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:05.226+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:05.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:05.264 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.264 INFO:teuthology.orchestra.run.smithi031.stdout:2902 read 0x31ccbc8 thru 0x31d52b4 (0x86ed bytes) 2024-09-13T07:13:05.266 INFO:teuthology.orchestra.run.smithi031.stdout:2903 trunc from 0xc7f9488 to 0xb773a2f 2024-09-13T07:13:05.282 INFO:teuthology.orchestra.run.smithi031.stdout:2905 trunc from 0xb773a2f to 0x23a17c7 2024-09-13T07:13:05.311 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:13:05.326 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:05.327+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:05.400 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:05.400 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:05.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:05.458 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:05.459+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:05.527 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:05.528+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.305852764000001, 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.304374739, 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.301557689999999, 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.732128729999999, 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.719505133999998, 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:13:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3179 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'130, mlcod=27'127)", 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:13:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.576222504, 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027322449999999999, 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027552+0000", 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027551+0000", 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027558+0000", 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027569+0000", 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.059e-05 2024-09-13T07:13:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027575+0000", 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5420000000000002e-06 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027617+0000", 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1328999999999997e-05 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.027637+0000", 2024-09-13T07:13:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0407999999999998e-05 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030260+0000", 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026228509999999998 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:41.030284+0000", 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4389999999999999e-05 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.975923366, 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:13:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:13:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.975386139000001, 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:13:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:13:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.836304503000001, 2024-09-13T07:13:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4162 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'113, mlcod=29'110)", 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.789496785000001, 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028573499999999998, 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814277+0000", 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814283+0000", 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814293+0000", 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0299000000000001e-05 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814298+0000", 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:13:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814345+0000", 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7707999999999998e-05 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814365+0000", 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9836000000000001e-05 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817100+0000", 2024-09-13T07:13:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027346969999999999 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817135+0000", 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5194999999999998e-05 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.78895721, 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.761473105, 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:13:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:13:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:13:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4195 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'122, mlcod=29'119)", 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.755937498000002, 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027697870000000001, 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847836+0000", 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847841+0000", 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847850+0000", 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2390000000000002e-06 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847854+0000", 2024-09-13T07:13:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847880+0000", 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847900+0000", 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0015000000000001e-05 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850568+0000", 2024-09-13T07:13:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002668283 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850607+0000", 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8349999999999997e-05 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.755458681, 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.703766387000002, 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:13:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.167542538999999, 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7730 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'61, mlcod=34'54)", 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.16664076, 2024-09-13T07:13:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028499979999999999, 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437133+0000", 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437139+0000", 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7409999999999998e-06 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437151+0000", 2024-09-13T07:13:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1946e-05 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437157+0000", 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437201+0000", 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3917e-05 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437221+0000", 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439958+0000", 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027367300000000002 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439984+0000", 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5551999999999999e-05 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8060 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'136, mlcod=34'133)", 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.840287976999999, 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002785984, 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763486+0000", 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763492+0000", 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5050000000000004e-06 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763510+0000", 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7833e-05 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763517+0000", 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763556+0000", 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8702999999999999e-05 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763577+0000", 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1375000000000001e-05 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766246+0000", 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026686890000000001 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766273+0000", 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6930000000000001e-05 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.407389361, 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8969806719999998, 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:05.692 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:05.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:2906 read 0x7ab5a0 thru 0x7b5db1 (0xa812 bytes) 2024-09-13T07:13:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:2907 writesame 0xb0ef33e thru 0xb0fa8fb data_size 0x2b(0xb5be bytes) 2024-09-13T07:13:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:2908 write 0x9a0cccc thru 0x9a170b0 (0xa3e5 bytes) 2024-09-13T07:13:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:2909 read 0x8827bca thru 0x882c936 (0x4d6d bytes) 2024-09-13T07:13:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:2910 trunc from 0xb0fa8fc to 0xb874f6e 2024-09-13T07:13:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:2911 trunc from 0xb874f6e to 0x99bdb7 2024-09-13T07:13:05.695 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:05.694+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:05.696 INFO:teuthology.orchestra.run.smithi031.stdout:2913 read 0x66a94f thru 0x672a95 (0x8147 bytes) 2024-09-13T07:13:05.697 INFO:teuthology.orchestra.run.smithi031.stdout:2914 punch from 0x15392 to 0x1f139, (0x9da7 bytes) 2024-09-13T07:13:05.697 INFO:teuthology.orchestra.run.smithi031.stdout:2915 read 0x38ca2e thru 0x38fd64 (0x3337 bytes) 2024-09-13T07:13:05.698 INFO:teuthology.orchestra.run.smithi031.stdout:2917 read 0x2e7040 thru 0x2eb526 (0x44e7 bytes) 2024-09-13T07:13:05.712 INFO:teuthology.orchestra.run.smithi031.stdout:2918 write 0x8c904e4 thru 0x8c924df (0x1ffc bytes) 2024-09-13T07:13:05.715 INFO:teuthology.orchestra.run.smithi031.stdout:2922 punch from 0x4b40865 to 0x4b50482, (0xfc1d bytes) 2024-09-13T07:13:05.715 INFO:teuthology.orchestra.run.smithi031.stdout:2923 punch from 0x2333f54 to 0x2334be8, (0xc94 bytes) 2024-09-13T07:13:05.716 INFO:teuthology.orchestra.run.smithi031.stdout:2924 trunc from 0x8c924e0 to 0x7fc3393 2024-09-13T07:13:05.722 INFO:teuthology.orchestra.run.smithi031.stdout:2926 read 0x384d6f8 thru 0x385d124 (0xfa2d bytes) 2024-09-13T07:13:05.734 INFO:teuthology.orchestra.run.smithi031.stdout:2927 write 0xe552410 thru 0xe558e3c (0x6a2d bytes) 2024-09-13T07:13:05.737 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.737 INFO:teuthology.orchestra.run.smithi031.stdout:2930 write 0xcac55d2 thru 0xcac5ae5 (0x514 bytes) 2024-09-13T07:13:05.740 INFO:teuthology.orchestra.run.smithi031.stdout:2931 write 0xdee3099 thru 0xdef2399 (0xf301 bytes) 2024-09-13T07:13:05.741 INFO:teuthology.orchestra.run.smithi031.stdout:2932 write 0x482fa19 thru 0x4833a0d (0x3ff5 bytes) 2024-09-13T07:13:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:2934 punch from 0x561ff92 to 0x5625cfb, (0x5d69 bytes) 2024-09-13T07:13:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:2935 write 0x86f84bd thru 0x8703e1c (0xb960 bytes) 2024-09-13T07:13:05.748 INFO:teuthology.orchestra.run.smithi031.stdout:2936 read 0x5e72a7a thru 0x5e76fa4 (0x452b bytes) 2024-09-13T07:13:05.748 INFO:teuthology.orchestra.run.smithi031.stdout:2937 read 0x4dea747 thru 0x4deaccd (0x587 bytes) 2024-09-13T07:13:05.748 INFO:teuthology.orchestra.run.smithi031.stdout:2938 read 0x59ee097 thru 0x59fcc77 (0xebe1 bytes) 2024-09-13T07:13:05.748 INFO:teuthology.orchestra.run.smithi031.stdout:2941 punch from 0x8683497 to 0x8685e28, (0x2991 bytes) 2024-09-13T07:13:05.749 INFO:teuthology.orchestra.run.smithi031.stdout:2943 trunc from 0xe558e3d to 0xd09256e 2024-09-13T07:13:05.763 INFO:teuthology.orchestra.run.smithi031.stdout:2944 trunc from 0xd09256e to 0x98e4c90 2024-09-13T07:13:05.773 INFO:teuthology.orchestra.run.smithi031.stdout:2945 trunc from 0x98e4c90 to 0x96d5b7d 2024-09-13T07:13:05.778 INFO:teuthology.orchestra.run.smithi031.stdout:2950 read 0x36d33fe thru 0x36dd571 (0xa174 bytes) 2024-09-13T07:13:05.779 INFO:teuthology.orchestra.run.smithi031.stdout:2951 read 0x8ee63ec thru 0x8ee64ee (0x103 bytes) 2024-09-13T07:13:05.784 INFO:teuthology.orchestra.run.smithi031.stdout:2952 trunc from 0x96d5b7d to 0xdf435e3 2024-09-13T07:13:05.790 INFO:teuthology.orchestra.run.smithi031.stdout:2954 write 0x4d8bb89 thru 0x4d93a62 (0x7eda bytes) 2024-09-13T07:13:05.793 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.794 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:05.794+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:05.796 INFO:teuthology.orchestra.run.smithi031.stdout:2956 trunc from 0xdf435e3 to 0xb4f2dca 2024-09-13T07:13:05.805 INFO:teuthology.orchestra.run.smithi031.stdout:2957 trunc from 0xb4f2dca to 0x7768155 2024-09-13T07:13:05.812 INFO:teuthology.orchestra.run.smithi031.stdout:2958 read 0x24a0a75 thru 0x24a6120 (0x56ac bytes) 2024-09-13T07:13:05.812 INFO:teuthology.orchestra.run.smithi031.stdout:2959 write 0x6880f57 thru 0x68885b9 (0x7663 bytes) 2024-09-13T07:13:05.814 INFO:teuthology.orchestra.run.smithi031.stdout:2960 read 0x640a05f thru 0x640c442 (0x23e4 bytes) 2024-09-13T07:13:05.815 INFO:teuthology.orchestra.run.smithi031.stdout:2962 punch from 0x5f1bda4 to 0x5f1e091, (0x22ed bytes) 2024-09-13T07:13:05.815 INFO:teuthology.orchestra.run.smithi031.stdout:2963 writesame 0x3d7f8b thru 0x3e3df0 data_size 0x1(0xbe66 bytes) 2024-09-13T07:13:05.819 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:05.819 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:05.819 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:05.819 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:05.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:05.832 INFO:teuthology.orchestra.run.smithi031.stdout:2964 trunc from 0x7768155 to 0x32727df 2024-09-13T07:13:05.843 INFO:teuthology.orchestra.run.smithi031.stdout:2966 punch from 0x1a9fea8 to 0x1aa60a3, (0x61fb bytes) 2024-09-13T07:13:05.843 INFO:teuthology.orchestra.run.smithi031.stdout:2967 write 0xd4d13e thru 0xd4dfa8 (0xe6b bytes) 2024-09-13T07:13:05.861 INFO:teuthology.orchestra.run.smithi031.stdout:2968 write 0xd46e130 thru 0xd47ad86 (0xcc57 bytes) 2024-09-13T07:13:05.863 INFO:teuthology.orchestra.run.smithi031.stdout:2969 read 0x1c4e7c3 thru 0x1c524e6 (0x3d24 bytes) 2024-09-13T07:13:05.864 INFO:teuthology.orchestra.run.smithi031.stdout:2970 write 0x1dd9705 thru 0x1de30e3 (0x99df bytes) 2024-09-13T07:13:05.865 INFO:teuthology.orchestra.run.smithi031.stdout:2972 punch from 0xa588d07 to 0xa597d1e, (0xf017 bytes) 2024-09-13T07:13:05.866 INFO:teuthology.orchestra.run.smithi031.stdout:2973 read 0x58f88b4 thru 0x5908238 (0xf985 bytes) 2024-09-13T07:13:05.866 INFO:teuthology.orchestra.run.smithi031.stdout:2974 read 0x453e13 thru 0x45efe6 (0xb1d4 bytes) 2024-09-13T07:13:05.866 INFO:teuthology.orchestra.run.smithi031.stdout:2975 read 0xa4cd02f thru 0xa4d29db (0x59ad bytes) 2024-09-13T07:13:05.880 INFO:teuthology.orchestra.run.smithi031.stdout:2976 trunc from 0xd47ad87 to 0xdc47b3 2024-09-13T07:13:05.894 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:05.895+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:05.901 INFO:teuthology.orchestra.run.smithi031.stdout:2977 trunc from 0xdc47b3 to 0x8b12c57 2024-09-13T07:13:05.906 INFO:teuthology.orchestra.run.smithi031.stdout:2978 read 0x6fc40a8 thru 0x6fc81b9 (0x4112 bytes) 2024-09-13T07:13:05.906 INFO:teuthology.orchestra.run.smithi031.stdout:2980 read 0x3f1eaa4 thru 0x3f21423 (0x2980 bytes) 2024-09-13T07:13:05.906 INFO:teuthology.orchestra.run.smithi031.stdout:2981 read 0x7f08173 thru 0x7f0e8b0 (0x673e bytes) 2024-09-13T07:13:05.906 INFO:teuthology.orchestra.run.smithi031.stdout:2983 write 0x61d92b4 thru 0x61ded2d (0x5a7a bytes) 2024-09-13T07:13:05.921 INFO:teuthology.orchestra.run.smithi031.stdout:2984 writesame 0xe68c502 thru 0xe69a4c8 data_size 0x1(0xdfc7 bytes) 2024-09-13T07:13:05.936 INFO:teuthology.orchestra.run.smithi031.stdout:2985 trunc from 0xe69a4c9 to 0x3683537 2024-09-13T07:13:05.946 INFO:teuthology.orchestra.run.smithi031.stdout:2986 read 0x144fa53 thru 0x1453ba2 (0x4150 bytes) 2024-09-13T07:13:05.947 INFO:teuthology.orchestra.run.smithi031.stdout:2987 read 0x259a307 thru 0x259c190 (0x1e8a bytes) 2024-09-13T07:13:05.947 INFO:teuthology.orchestra.run.smithi031.stdout:2988 read 0x5205e8 thru 0x52214a (0x1b63 bytes) 2024-09-13T07:13:05.947 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.948 INFO:teuthology.orchestra.run.smithi031.stdout:2992 punch from 0x1b324eb to 0x1b3af13, (0x8a28 bytes) 2024-09-13T07:13:05.957 INFO:teuthology.orchestra.run.smithi031.stdout:2994 writesame 0x7718149 thru 0x7727f2b data_size 0x5(0xfde3 bytes) 2024-09-13T07:13:05.957 INFO:teuthology.orchestra.run.smithi031.stdout:2996 write 0x7616f2c thru 0x7624d9d (0xde72 bytes) 2024-09-13T07:13:05.961 INFO:teuthology.orchestra.run.smithi031.stdout:2998 writesame 0x666e1b0 thru 0x66757d9 data_size 0x37(0x762a bytes) 2024-09-13T07:13:05.963 INFO:teuthology.orchestra.run.smithi031.stdout:2999 punch from 0x24f16e7 to 0x24fe193, (0xcaac bytes) 2024-09-13T07:13:05.963 INFO:teuthology.orchestra.run.smithi031.stdout:3002 write 0x99bd38 thru 0x99bf84 (0x24d bytes) 2024-09-13T07:13:05.970 INFO:teuthology.orchestra.run.smithi031.stdout:3004 write 0x8a6ed67 thru 0x8a7912c (0xa3c6 bytes) 2024-09-13T07:13:05.973 INFO:teuthology.orchestra.run.smithi031.stdout:3005 punch from 0x142da90 to 0x143b1ea, (0xd75a bytes) 2024-09-13T07:13:05.974 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:05.974 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:05.984 INFO:teuthology.orchestra.run.smithi031.stdout:3006 write 0xe08e201 thru 0xe09da56 (0xf856 bytes) 2024-09-13T07:13:05.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:05.987 INFO:teuthology.orchestra.run.smithi031.stdout:3008 write 0x149d8c8 thru 0x14a33b2 (0x5aeb bytes) 2024-09-13T07:13:05.989 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.989 INFO:teuthology.orchestra.run.smithi031.stdout:3010 read 0xb531f11 thru 0xb5320dc (0x1cc bytes) 2024-09-13T07:13:05.989 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.989 INFO:teuthology.orchestra.run.smithi031.stdout:3014 punch from 0x56b861 to 0x575e9b, (0xa63a bytes) 2024-09-13T07:13:05.990 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:05.999 INFO:teuthology.orchestra.run.smithi031.stdout:3018 trunc from 0xe09da57 to 0x496b7a1 2024-09-13T07:13:05.999 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:06.000+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:06.020 INFO:teuthology.orchestra.run.smithi031.stdout:3019 write 0x9ea8a40 thru 0x9ead544 (0x4b05 bytes) 2024-09-13T07:13:06.023 INFO:teuthology.orchestra.run.smithi031.stdout:3020 punch from 0xe27104 to 0xe31d2f, (0xac2b bytes) 2024-09-13T07:13:06.023 INFO:teuthology.orchestra.run.smithi031.stdout:3021 write 0x3763995 thru 0x376bbe3 (0x824f bytes) 2024-09-13T07:13:06.024 INFO:teuthology.orchestra.run.smithi031.stdout:3023 write 0x2a4652a thru 0x2a4e32e (0x7e05 bytes) 2024-09-13T07:13:06.026 INFO:teuthology.orchestra.run.smithi031.stdout:3024 write 0x4197813 thru 0x41a491a (0xd108 bytes) 2024-09-13T07:13:06.034 INFO:teuthology.orchestra.run.smithi031.stdout:3025 writesame 0xa8f5407 thru 0xa8f578b data_size 0x1(0x385 bytes) 2024-09-13T07:13:06.040 INFO:teuthology.orchestra.run.smithi031.stdout:3027 trunc from 0xa8f578c to 0x5b788bc 2024-09-13T07:13:06.050 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.059 INFO:teuthology.orchestra.run.smithi031.stdout:3030 write 0x954ded8 thru 0x954e7ee (0x917 bytes) 2024-09-13T07:13:06.061 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.065 INFO:teuthology.orchestra.run.smithi031.stdout:3032 trunc from 0x954e7ef to 0x614ff1f 2024-09-13T07:13:06.074 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.079 INFO:teuthology.orchestra.run.smithi031.stdout:3035 trunc from 0x614ff1f to 0xa0d17b5 2024-09-13T07:13:06.093 INFO:teuthology.orchestra.run.smithi031.stdout:3036 trunc from 0xa0d17b5 to 0x183413d 2024-09-13T07:13:06.096 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:06.097+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:06.103 INFO:teuthology.orchestra.run.smithi031.stdout:3037 read 0xe8a1d2 thru 0xe9872d (0xe55c bytes) 2024-09-13T07:13:06.116 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5548154269999999, 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5547901130000001, 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:13:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:13:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554755747, 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:13:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554726923, 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:13:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554708626, 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:13:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5546838200000002, 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:13:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5546659470000002, 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:13:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:13:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:13:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5546396059999998, 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:13:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554612884, 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:13:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:13:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554595299, 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554568661, 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:13:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554551201, 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5545242429999999, 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:13:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:13:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:13:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5544980390000001, 2024-09-13T07:13:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:13:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:13:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:13:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:13:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:13:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:13:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554480047, 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:13:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.55438828, 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:13:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:13:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:13:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.55436598, 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:13:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5543349790000001, 2024-09-13T07:13:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5543102229999999, 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:13:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:13:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.554292531, 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:13:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:13:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:06.219 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:06.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:06.220 INFO:teuthology.orchestra.run.smithi031.stdout:3039 writesame 0xec0fe40 thru 0xec1f84c data_size 0x1(0xfa0d bytes) 2024-09-13T07:13:06.221 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:06.223 INFO:teuthology.orchestra.run.smithi031.stdout:3040 trunc from 0xec1f84d to 0x6d98397 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3041 write 0xab03218 thru 0xab0651b (0x3304 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3042 read 0x9e6fa41 thru 0x9e7a0ed (0xa6ad bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3044 read 0x3ae99be thru 0x3af6770 (0xcdb3 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3046 punch from 0x2decc59 to 0x2df3777, (0x6b1e bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3047 write 0xc39816b thru 0xc3a1715 (0x95ab bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3048 read 0x65f019e thru 0x65f84ff (0x8362 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3049 write 0x603358c thru 0x6042b9a (0xf60f bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3050 read 0x378e27 thru 0x37c382 (0x355c bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3051 read 0x5acc7ee thru 0x5ad65c5 (0x9dd8 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3052 read 0x203e4a6 thru 0x2045f4e (0x7aa9 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3053 write 0x736ded6 thru 0x736e6b6 (0x7e1 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3055 writesame 0x494625a thru 0x4947c7c data_size 0x1(0x1a23 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3056 read 0x30c90f5 thru 0x30ce5db (0x54e7 bytes) 2024-09-13T07:13:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:3057 writesame 0xe536502 thru 0xe545ba8 data_size 0x1(0xf6a7 bytes) 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3059 writesame 0x250efb9 thru 0x251bf8c data_size 0x5e(0xcfd4 bytes) 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3060 write 0x470c87a thru 0x4710ecd (0x4654 bytes) 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3063 writesame 0x4fc9919 thru 0x4fce45e data_size 0x52(0x4b46 bytes) 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3064 writesame 0x323db91 thru 0x323f742 data_size 0xa(0x1bb2 bytes) 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3065 punch from 0x9bad5ce to 0x9bbc569, (0xef9b bytes) 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3066 read 0xdd2ffeb thru 0xdd33640 (0x3656 bytes) 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3067 trunc from 0xe545ba9 to 0x2aaac29 2024-09-13T07:13:06.225 INFO:teuthology.orchestra.run.smithi031.stdout:3068 read 0xfa1954 thru 0xfa9b76 (0x8223 bytes) 2024-09-13T07:13:06.226 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:06.220+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:06.235 INFO:teuthology.orchestra.run.smithi031.stdout:3069 writesame 0x9e47db2 thru 0x9e4d72c data_size 0x1(0x597b bytes) 2024-09-13T07:13:06.235 INFO:teuthology.orchestra.run.smithi031.stdout:3072 write 0x95777bd thru 0x9579e66 (0x26aa bytes) 2024-09-13T07:13:06.237 INFO:teuthology.orchestra.run.smithi031.stdout:3073 read 0x45a12e6 thru 0x45b11db (0xfef6 bytes) 2024-09-13T07:13:06.237 INFO:teuthology.orchestra.run.smithi031.stdout:3076 write 0x1f938f1 thru 0x1f9cf42 (0x9652 bytes) 2024-09-13T07:13:06.242 INFO:teuthology.orchestra.run.smithi031.stdout:3078 trunc from 0x9e4d72d to 0x6ceb3c5 2024-09-13T07:13:06.264 INFO:teuthology.orchestra.run.smithi031.stdout:3080 write 0xe1f18c1 thru 0xe1f3228 (0x1968 bytes) 2024-09-13T07:13:06.267 INFO:teuthology.orchestra.run.smithi031.stdout:3083 write 0x3f7841a thru 0x3f7ddc9 (0x59b0 bytes) 2024-09-13T07:13:06.269 INFO:teuthology.orchestra.run.smithi031.stdout:3084 write 0x16697e3 thru 0x16759d4 (0xc1f2 bytes) 2024-09-13T07:13:06.270 INFO:teuthology.orchestra.run.smithi031.stdout:3086 read 0x11fde56 thru 0x1206477 (0x8622 bytes) 2024-09-13T07:13:06.277 INFO:teuthology.orchestra.run.smithi031.stdout:3087 write 0xec3f98d thru 0xec41f83 (0x25f7 bytes) 2024-09-13T07:13:06.279 INFO:teuthology.orchestra.run.smithi031.stdout:3088 writesame 0x92e7c99 thru 0x92e9bf0 data_size 0x3b(0x1f58 bytes) 2024-09-13T07:13:06.282 INFO:teuthology.orchestra.run.smithi031.stdout:3089 punch from 0x51e3d50 to 0x51ea46c, (0x671c bytes) 2024-09-13T07:13:06.282 INFO:teuthology.orchestra.run.smithi031.stdout:3091 writesame 0xa100ac5 thru 0xa10a5b3 data_size 0x27(0x9aef bytes) 2024-09-13T07:13:06.284 INFO:teuthology.orchestra.run.smithi031.stdout:3092 read 0x74bce6 thru 0x751ed6 (0x61f1 bytes) 2024-09-13T07:13:06.285 INFO:teuthology.orchestra.run.smithi031.stdout:3093 write 0x7f7c0df thru 0x7f7ffbe (0x3ee0 bytes) 2024-09-13T07:13:06.287 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.288 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.288 INFO:teuthology.orchestra.run.smithi031.stdout:3097 write 0xa078070 thru 0xa086b2c (0xeabd bytes) 2024-09-13T07:13:06.291 INFO:teuthology.orchestra.run.smithi031.stdout:3098 read 0x616cb90 thru 0x617ab4d (0xdfbe bytes) 2024-09-13T07:13:06.306 INFO:teuthology.orchestra.run.smithi031.stdout:3099 trunc from 0xec41f84 to 0x2f6a124 2024-09-13T07:13:06.320 INFO:teuthology.orchestra.run.smithi031.stdout:3100 read 0xc7b6eb thru 0xc83763 (0x8079 bytes) 2024-09-13T07:13:06.321 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:06.322+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:06.325 INFO:teuthology.orchestra.run.smithi031.stdout:3101 trunc from 0x2f6a124 to 0x8220d63 2024-09-13T07:13:06.336 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:06.336 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:06.336 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:06.336 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:06.338 INFO:teuthology.orchestra.run.smithi031.stdout:3102 write 0xb2c58b8 thru 0xb2c9a8d (0x41d6 bytes) 2024-09-13T07:13:06.341 INFO:teuthology.orchestra.run.smithi031.stdout:3103 read 0x5dd4e08 thru 0x5dd4f91 (0x18a bytes) 2024-09-13T07:13:06.341 INFO:teuthology.orchestra.run.smithi031.stdout:3104 write 0x6a27b51 thru 0x6a33254 (0xb704 bytes) 2024-09-13T07:13:06.343 INFO:teuthology.orchestra.run.smithi031.stdout:3106 read 0x838db6a thru 0x8392c9d (0x5134 bytes) 2024-09-13T07:13:06.344 INFO:teuthology.orchestra.run.smithi031.stdout:3107 write 0x6ee848d thru 0x6ef0954 (0x84c8 bytes) 2024-09-13T07:13:06.346 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:06.347 INFO:teuthology.orchestra.run.smithi031.stdout:3108 read 0x6980c0b thru 0x69842a5 (0x369b bytes) 2024-09-13T07:13:06.347 INFO:teuthology.orchestra.run.smithi031.stdout:3109 punch from 0x3436ac0 to 0x34404a3, (0x99e3 bytes) 2024-09-13T07:13:06.347 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.347 INFO:teuthology.orchestra.run.smithi031.stdout:3111 read 0x9bfd5d6 thru 0x9c0c86c (0xf297 bytes) 2024-09-13T07:13:06.347 INFO:teuthology.orchestra.run.smithi031.stdout:3112 read 0x7b56a2d thru 0x7b57522 (0xaf6 bytes) 2024-09-13T07:13:06.347 INFO:teuthology.orchestra.run.smithi031.stdout:3114 read 0x149c6da thru 0x149dcc7 (0x15ee bytes) 2024-09-13T07:13:06.347 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.358 INFO:teuthology.orchestra.run.smithi031.stdout:3116 trunc from 0xb2c9a8e to 0x2629670 2024-09-13T07:13:06.369 INFO:teuthology.orchestra.run.smithi031.stdout:3117 read 0x1a7eeb2 thru 0x1a87812 (0x8961 bytes) 2024-09-13T07:13:06.370 INFO:teuthology.orchestra.run.smithi031.stdout:3118 trunc from 0x2629670 to 0x3912b8e 2024-09-13T07:13:06.376 INFO:teuthology.orchestra.run.smithi031.stdout:3120 read 0x8fcba0 thru 0x90b7db (0xec3c bytes) 2024-09-13T07:13:06.376 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.392 INFO:teuthology.orchestra.run.smithi031.stdout:3123 writesame 0xd43be58 thru 0xd43db71 data_size 0x19(0x1d1a bytes) 2024-09-13T07:13:06.392 INFO:teuthology.orchestra.run.smithi031.stdout:3124 write 0x2eed920 thru 0x2ef1efa (0x45db bytes) 2024-09-13T07:13:06.393 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:06.405 INFO:teuthology.orchestra.run.smithi031.stdout:3126 trunc from 0xd43db72 to 0x2c534fa 2024-09-13T07:13:06.415 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:06.415 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:06.422 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:06.422+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:06.428 INFO:teuthology.orchestra.run.smithi031.stdout:3127 write 0xd0eb006 thru 0xd0f974b (0xe746 bytes) 2024-09-13T07:13:06.430 INFO:teuthology.orchestra.run.smithi031.stdout:3128 read 0x8d60d94 thru 0x8d64437 (0x36a4 bytes) 2024-09-13T07:13:06.430 INFO:teuthology.orchestra.run.smithi031.stdout:3129 read 0x87eeed5 thru 0x87f9f32 (0xb05e bytes) 2024-09-13T07:13:06.433 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:06.437 INFO:teuthology.orchestra.run.smithi031.stdout:3132 trunc from 0xd0f974c to 0x69dee35 2024-09-13T07:13:06.452 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:06.453 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:06.454 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:06.454 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:06.454 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:06.454 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:06.454 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:06.454 INFO:teuthology.orchestra.run.smithi031.stdout:3133 write 0x9c2f3a1 thru 0x9c31620 (0x2280 bytes) 2024-09-13T07:13:06.456 INFO:teuthology.orchestra.run.smithi031.stdout:3134 punch from 0x61feca0 to 0x620e8d4, (0xfc34 bytes) 2024-09-13T07:13:06.457 INFO:teuthology.orchestra.run.smithi031.stdout:3135 read 0x95ae02c thru 0x95ba020 (0xbff5 bytes) 2024-09-13T07:13:06.457 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.461 INFO:teuthology.orchestra.run.smithi031.stdout:3137 trunc from 0x9c31621 to 0xe239f48 2024-09-13T07:13:06.463 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:06.466 INFO:teuthology.orchestra.run.smithi031.stdout:3138 read 0x9059112 thru 0x90651d6 (0xc0c5 bytes) 2024-09-13T07:13:06.466 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.466 INFO:teuthology.orchestra.run.smithi031.stdout:3140 writesame 0x459df thru 0x4bf71 data_size 0x1(0x6593 bytes) 2024-09-13T07:13:06.470 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:06.479 INFO:teuthology.orchestra.run.smithi031.stdout:3141 write 0x116a0a5 thru 0x1179e3e (0xfd9a bytes) 2024-09-13T07:13:06.481 INFO:teuthology.orchestra.run.smithi031.stdout:3142 trunc from 0xe239f48 to 0xd841435 2024-09-13T07:13:06.485 INFO:teuthology.orchestra.run.smithi031.stdout:3143 write 0x6e32b26 thru 0x6e3434d (0x1828 bytes) 2024-09-13T07:13:06.488 INFO:teuthology.orchestra.run.smithi031.stdout:3144 write 0x489444b thru 0x4898602 (0x41b8 bytes) 2024-09-13T07:13:06.488 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:06.488 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:06.490 INFO:teuthology.orchestra.run.smithi031.stdout:3145 punch from 0x8a11723 to 0x8a11ae0, (0x3bd bytes) 2024-09-13T07:13:06.490 INFO:teuthology.orchestra.run.smithi031.stdout:3146 read 0xacc0627 thru 0xaccf05d (0xea37 bytes) 2024-09-13T07:13:06.490 INFO:teuthology.orchestra.run.smithi031.stdout:3147 punch from 0x24a097e to 0x24ab927, (0xafa9 bytes) 2024-09-13T07:13:06.490 INFO:teuthology.orchestra.run.smithi031.stdout:3148 read 0x3f47aaf thru 0x3f48fbf (0x1511 bytes) 2024-09-13T07:13:06.490 INFO:teuthology.orchestra.run.smithi031.stdout:3150 read 0xaf54088 thru 0xaf60805 (0xc77e bytes) 2024-09-13T07:13:06.490 INFO:teuthology.orchestra.run.smithi031.stdout:3151 writesame 0xcbec6b8 thru 0xcbfc59b data_size 0x16(0xfee4 bytes) 2024-09-13T07:13:06.493 INFO:teuthology.orchestra.run.smithi031.stdout:3152 punch from 0x519a7ca to 0x51a7169, (0xc99f bytes) 2024-09-13T07:13:06.493 INFO:teuthology.orchestra.run.smithi031.stdout:3155 write 0x1498237 thru 0x149b731 (0x34fb bytes) 2024-09-13T07:13:06.494 INFO:teuthology.orchestra.run.smithi031.stdout:3157 read 0x83f9a69 thru 0x84091d2 (0xf76a bytes) 2024-09-13T07:13:06.494 INFO:teuthology.orchestra.run.smithi031.stdout:3160 read 0x1669143 thru 0x166c581 (0x343f bytes) 2024-09-13T07:13:06.494 INFO:teuthology.orchestra.run.smithi031.stdout:3161 punch from 0x2a90c0b to 0x2a97dc9, (0x71be bytes) 2024-09-13T07:13:06.498 INFO:teuthology.orchestra.run.smithi031.stdout:3163 trunc from 0xd841435 to 0xa982824 2024-09-13T07:13:06.504 INFO:teuthology.orchestra.run.smithi031.stdout:3165 punch from 0x2175d2d to 0x217b94c, (0x5c1f bytes) 2024-09-13T07:13:06.507 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:13:06.508 INFO:teuthology.orchestra.run.smithi031.stdout:3166 write 0xad3dfc1 thru 0xad43800 (0x5840 bytes) 2024-09-13T07:13:06.511 INFO:teuthology.orchestra.run.smithi031.stdout:3167 read 0x66b9925 thru 0x66c2b95 (0x9271 bytes) 2024-09-13T07:13:06.511 INFO:teuthology.orchestra.run.smithi031.stdout:3169 punch from 0x26ffbfb to 0x2707b3f, (0x7f44 bytes) 2024-09-13T07:13:06.512 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:13:06.512 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:13:06.512 INFO:teuthology.orchestra.run.smithi031.stdout:3170 trunc from 0xad43801 to 0xb6ad612 2024-09-13T07:13:06.516 INFO:teuthology.orchestra.run.smithi031.stdout:3171 read 0xafa23eb thru 0xafacff7 (0xac0d bytes) 2024-09-13T07:13:06.516 INFO:teuthology.orchestra.run.smithi031.stdout:3173 write 0x8e27a36 thru 0x8e3330d (0xb8d8 bytes) 2024-09-13T07:13:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:3174 read 0x13168bf thru 0x13209fd (0xa13f bytes) 2024-09-13T07:13:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:3175 write 0x813026f thru 0x8135f04 (0x5c96 bytes) 2024-09-13T07:13:06.520 INFO:teuthology.orchestra.run.smithi031.stdout:3176 read 0x3426ba2 thru 0x343255a (0xb9b9 bytes) 2024-09-13T07:13:06.521 INFO:teuthology.orchestra.run.smithi031.stdout:3177 read 0x9f46b50 thru 0x9f56401 (0xf8b2 bytes) 2024-09-13T07:13:06.524 INFO:teuthology.orchestra.run.smithi031.stdout:3180 read 0x3038df0 thru 0x3044edf (0xc0f0 bytes) 2024-09-13T07:13:06.525 INFO:teuthology.orchestra.run.smithi031.stdout:3181 read 0x93de292 thru 0x93e0fd2 (0x2d41 bytes) 2024-09-13T07:13:06.525 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.526 INFO:teuthology.orchestra.run.smithi031.stdout:3183 writesame 0x3e19a6c thru 0x3e2111a data_size 0x1(0x76af bytes) 2024-09-13T07:13:06.526 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:06.526+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:06.533 INFO:teuthology.orchestra.run.smithi031.stdout:3184 read 0x6c6447b thru 0x6c6f35b (0xaee1 bytes) 2024-09-13T07:13:06.540 INFO:teuthology.orchestra.run.smithi031.stdout:3185 write 0xc3db0a4 thru 0xc3e4359 (0x92b6 bytes) 2024-09-13T07:13:06.542 INFO:teuthology.orchestra.run.smithi031.stdout:3187 writesame 0xa2f5116 thru 0xa2fb853 data_size 0xf(0x673e bytes) 2024-09-13T07:13:06.545 INFO:teuthology.orchestra.run.smithi031.stdout:3188 read 0x35a5d2 thru 0x365388 (0xadb7 bytes) 2024-09-13T07:13:06.546 INFO:teuthology.orchestra.run.smithi031.stdout:3189 read 0xc119c31 thru 0xc121c0f (0x7fdf bytes) 2024-09-13T07:13:06.546 INFO:teuthology.orchestra.run.smithi031.stdout:3190 write 0x4639d6c thru 0x46475ae (0xd843 bytes) 2024-09-13T07:13:06.550 INFO:teuthology.orchestra.run.smithi031.stdout:3192 read 0x7791dbc thru 0x7795407 (0x364c bytes) 2024-09-13T07:13:06.559 INFO:teuthology.orchestra.run.smithi031.stdout:3194 write 0xe583f01 thru 0xe584cd1 (0xdd1 bytes) 2024-09-13T07:13:06.561 INFO:teuthology.orchestra.run.smithi031.stdout:3196 write 0xb5a7d50 thru 0xb5a9588 (0x1839 bytes) 2024-09-13T07:13:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:3198 punch from 0x98c2c64 to 0x98cc863, (0x9bff bytes) 2024-09-13T07:13:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.566 INFO:teuthology.orchestra.run.smithi031.stdout:3203 trunc from 0xe584cd2 to 0xb19cbe4 2024-09-13T07:13:06.577 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.577 INFO:teuthology.orchestra.run.smithi031.stdout:3205 punch from 0x9b2a55 to 0x9bd527, (0xaad2 bytes) 2024-09-13T07:13:06.577 INFO:teuthology.orchestra.run.smithi031.stdout:3207 read 0xaa87ece thru 0xaa8927a (0x13ad bytes) 2024-09-13T07:13:06.587 INFO:teuthology.orchestra.run.smithi031.stdout:3208 writesame 0xe4280c0 thru 0xe4283cf data_size 0xe(0x310 bytes) 2024-09-13T07:13:06.587 INFO:teuthology.orchestra.run.smithi031.stdout:3210 read 0xa3324d thru 0xa39b98 (0x694c bytes) 2024-09-13T07:13:06.587 INFO:teuthology.orchestra.run.smithi031.stdout:3211 read 0x1740264 thru 0x1741c0d (0x19aa bytes) 2024-09-13T07:13:06.587 INFO:teuthology.orchestra.run.smithi031.stdout:3213 write 0x1969839 thru 0x196f9af (0x6177 bytes) 2024-09-13T07:13:06.589 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.589 INFO:teuthology.orchestra.run.smithi031.stdout:3216 write 0x8c1de25 thru 0x8c2b3f5 (0xd5d1 bytes) 2024-09-13T07:13:06.592 INFO:teuthology.orchestra.run.smithi031.stdout:3218 write 0xb25fd71 thru 0xb266a2d (0x6cbd bytes) 2024-09-13T07:13:06.593 INFO:teuthology.orchestra.run.smithi031.stdout:3219 punch from 0xc1a68fa to 0xc1b2560, (0xbc66 bytes) 2024-09-13T07:13:06.593 INFO:teuthology.orchestra.run.smithi031.stdout:3220 writesame 0x23b0c5d thru 0x23b7e9e data_size 0x4b(0x7242 bytes) 2024-09-13T07:13:06.595 INFO:teuthology.orchestra.run.smithi031.stdout:3221 punch from 0xd82923f to 0xd83378f, (0xa550 bytes) 2024-09-13T07:13:06.595 INFO:teuthology.orchestra.run.smithi031.stdout:3222 writesame 0x547e28f thru 0x548d6d9 data_size 0x1(0xf44b bytes) 2024-09-13T07:13:06.611 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:06.611 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:06.620 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:06.623 INFO:teuthology.orchestra.run.smithi031.stdout:3223 write 0x6d3c8e6 thru 0x6d43837 (0x6f52 bytes) 2024-09-13T07:13:06.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.625 INFO:teuthology.orchestra.run.smithi031.stdout:3225 write 0xc3cad6d thru 0xc3cdf1f (0x31b3 bytes) 2024-09-13T07:13:06.626 INFO:teuthology.orchestra.run.smithi031.stdout:3226 read 0x2816421 thru 0x281df2c (0x7b0c bytes) 2024-09-13T07:13:06.626 INFO:teuthology.orchestra.run.smithi031.stdout:3227 write 0x9e968a thru 0x9e99ce (0x345 bytes) 2024-09-13T07:13:06.628 INFO:teuthology.orchestra.run.smithi031.stdout:3228 read 0xc1c029c thru 0xc1c6631 (0x6396 bytes) 2024-09-13T07:13:06.629 INFO:teuthology.orchestra.run.smithi031.stdout:3229 write 0x270e86f thru 0x271d2df (0xea71 bytes) 2024-09-13T07:13:06.630 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.630 INFO:teuthology.orchestra.run.smithi031.stdout:3231 write 0x4aa4f1b thru 0x4aac056 (0x713c bytes) 2024-09-13T07:13:06.631 INFO:teuthology.orchestra.run.smithi031.stdout:3232 write 0xcde31e6 thru 0xcde4113 (0xf2e bytes) 2024-09-13T07:13:06.633 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.634 INFO:teuthology.orchestra.run.smithi031.stdout:3234 writesame 0x4c5562 thru 0x4cb671 data_size 0x8(0x6110 bytes) 2024-09-13T07:13:06.650 INFO:teuthology.orchestra.run.smithi031.stdout:3235 trunc from 0xe4283d0 to 0x962a90 2024-09-13T07:13:06.664 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:06.664+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:06.683 INFO:teuthology.orchestra.run.smithi031.stdout:3236 write 0xdb3ee38 thru 0xdb40f05 (0x20ce bytes) 2024-09-13T07:13:06.686 INFO:teuthology.orchestra.run.smithi031.stdout:3237 read 0xcbadf54 thru 0xcbb8c47 (0xacf4 bytes) 2024-09-13T07:13:06.686 INFO:teuthology.orchestra.run.smithi031.stdout:3238 punch from 0x2175a81 to 0x217c256, (0x67d5 bytes) 2024-09-13T07:13:06.687 INFO:teuthology.orchestra.run.smithi031.stdout:3239 write 0x6db149a thru 0x6dbf94f (0xe4b6 bytes) 2024-09-13T07:13:06.693 INFO:teuthology.orchestra.run.smithi031.stdout:3240 trunc from 0xdb40f06 to 0xa578c88 2024-09-13T07:13:06.701 INFO:teuthology.orchestra.run.smithi031.stdout:3241 writesame 0x3c0ad5e thru 0x3c17baf data_size 0x3(0xce52 bytes) 2024-09-13T07:13:06.711 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.711 INFO:teuthology.orchestra.run.smithi031.stdout:3244 read 0x9835590 thru 0x9841f37 (0xc9a8 bytes) 2024-09-13T07:13:06.711 INFO:teuthology.orchestra.run.smithi031.stdout:3245 writesame 0x73d8f28 thru 0x73da7f9 data_size 0x9(0x18d2 bytes) 2024-09-13T07:13:06.714 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.714 INFO:teuthology.orchestra.run.smithi031.stdout:3247 read 0x1fdd11d thru 0x1fe2c46 (0x5b2a bytes) 2024-09-13T07:13:06.715 INFO:teuthology.orchestra.run.smithi031.stdout:3248 write 0x67736a4 thru 0x6779ff7 (0x6954 bytes) 2024-09-13T07:13:06.717 INFO:teuthology.orchestra.run.smithi031.stdout:3250 read 0x882023e thru 0x882d893 (0xd656 bytes) 2024-09-13T07:13:06.717 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.725 INFO:teuthology.orchestra.run.smithi031.stdout:3253 trunc from 0xa578c88 to 0x31f06cd 2024-09-13T07:13:06.726 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:06.727+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:06.727 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:06.727 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:06.737 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:06.740 INFO:teuthology.orchestra.run.smithi031.stdout:3254 trunc from 0x31f06cd to 0x199ba 2024-09-13T07:13:06.765 INFO:teuthology.orchestra.run.smithi031.stdout:3255 writesame 0x40c8ce5 thru 0x40d59fc data_size 0x4(0xcd18 bytes) 2024-09-13T07:13:06.771 INFO:teuthology.orchestra.run.smithi031.stdout:3256 writesame 0x6220278 thru 0x6223524 data_size 0x1(0x32ad bytes) 2024-09-13T07:13:06.771 INFO:teuthology.orchestra.run.smithi031.stdout:3258 write 0x46d6035 thru 0x46de0d7 (0x80a3 bytes) 2024-09-13T07:13:06.774 INFO:teuthology.orchestra.run.smithi031.stdout:3260 write 0x2c73faa thru 0x2c760cd (0x2124 bytes) 2024-09-13T07:13:06.777 INFO:teuthology.orchestra.run.smithi031.stdout:3261 read 0x522cda6 thru 0x52362a0 (0x94fb bytes) 2024-09-13T07:13:06.783 INFO:teuthology.orchestra.run.smithi031.stdout:3262 trunc from 0x6223525 to 0x624d92 2024-09-13T07:13:06.793 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.808 INFO:teuthology.orchestra.run.smithi031.stdout:3265 write 0x948a237 thru 0x948be79 (0x1c43 bytes) 2024-09-13T07:13:06.810 INFO:teuthology.orchestra.run.smithi031.stdout:3266 read 0x76d33bd thru 0x76d4b9d (0x17e1 bytes) 2024-09-13T07:13:06.811 INFO:teuthology.orchestra.run.smithi031.stdout:3267 read 0x9164278 thru 0x9171d26 (0xdaaf bytes) 2024-09-13T07:13:06.811 INFO:teuthology.orchestra.run.smithi031.stdout:3268 write 0x619643c thru 0x6199123 (0x2ce8 bytes) 2024-09-13T07:13:06.813 INFO:teuthology.orchestra.run.smithi031.stdout:3269 write 0x1161fc7 thru 0x11719e6 (0xfa20 bytes) 2024-09-13T07:13:06.816 INFO:teuthology.orchestra.run.smithi031.stdout:3270 punch from 0xac32ef to 0xacc971, (0x9682 bytes) 2024-09-13T07:13:06.816 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.816 INFO:teuthology.orchestra.run.smithi031.stdout:3272 read 0x34a3872 thru 0x34a9f74 (0x6703 bytes) 2024-09-13T07:13:06.826 INFO:teuthology.orchestra.run.smithi031.stdout:3275 write 0xd2fada3 thru 0xd2fc096 (0x12f4 bytes) 2024-09-13T07:13:06.826 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:06.827+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:06.836 INFO:teuthology.orchestra.run.smithi031.stdout:3276 trunc from 0xd2fc097 to 0x65a60f7 2024-09-13T07:13:06.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:06.846 INFO:teuthology.orchestra.run.smithi031.stdout:3278 clone 20 order 20 su 16384 sc 14 2024-09-13T07:13:06.882 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:06.882 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:06.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:06.928 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:06.928+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:07.028 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:07.029+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:07.034 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:07.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:07.129 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:07.128+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:07.144 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:07.145 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:07.153 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:07.230 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:07.230+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:07.295 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:07.296 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:07.304 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:07.330 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:07.331+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:07.347 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:13:07.402 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone19 from 0x65a60f7 (overlap 0x199ba) to 0x147c8 2024-09-13T07:13:07.409 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:07.410 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:07.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:07.431 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:07.432+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:07.487 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T07:13:07.533 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:07.534+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:07.565 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:07.565 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:07.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:07.662 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:07.663+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:07.704 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-13T07:13:07.704 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:07.704 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2024-09-13T07:13:07.704 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T07:13:07.704 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T07:13:07.707 INFO:tasks.ceph.osd.7:Started 2024-09-13T07:13:07.707 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:07.752 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:07.752+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:07.767 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:07.768 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:07.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:07.806 INFO:teuthology.orchestra.run.smithi184.stderr:no valid command found; 10 closest matches: 2024-09-13T07:13:07.806 INFO:teuthology.orchestra.run.smithi184.stderr:0 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:1 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:2 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:abort 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:assert 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:config diff 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:config diff get 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:config get 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:config help [] 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:config set ... 2024-09-13T07:13:07.807 INFO:teuthology.orchestra.run.smithi184.stderr:admin_socket: invalid command 2024-09-13T07:13:07.808 INFO:teuthology.orchestra.run.smithi031.stdout:3279 writesame 0x70a5bf6 thru 0x70a66ae data_size 0xf(0xab9 bytes) 2024-09-13T07:13:07.808 INFO:teuthology.orchestra.run.smithi031.stdout:3281 write 0x14e6edb thru 0x14e8c09 (0x1d2f bytes) 2024-09-13T07:13:07.809 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T07:13:07.809 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-09-13T07:13:07.824 INFO:teuthology.orchestra.run.smithi031.stdout:3282 write 0x8c917aa thru 0x8c9b52c (0x9d83 bytes) 2024-09-13T07:13:07.828 INFO:teuthology.orchestra.run.smithi031.stdout:3283 punch from 0x63643bd to 0x6367c70, (0x38b3 bytes) 2024-09-13T07:13:07.829 INFO:teuthology.orchestra.run.smithi031.stdout:3284 punch from 0x5dd2675 to 0x5dd9ff8, (0x7983 bytes) 2024-09-13T07:13:07.829 INFO:teuthology.orchestra.run.smithi031.stdout:3285 read 0x1043174 thru 0x104941f (0x62ac bytes) 2024-09-13T07:13:07.829 INFO:teuthology.orchestra.run.smithi031.stdout:3286 punch from 0x77fdfdb to 0x7809441, (0xb466 bytes) 2024-09-13T07:13:07.835 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:07.836+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:07.839 INFO:teuthology.orchestra.run.smithi031.stdout:3288 write 0xe2ec1d0 thru 0xe2ef2c4 (0x30f5 bytes) 2024-09-13T07:13:07.842 INFO:teuthology.orchestra.run.smithi031.stdout:3289 read 0x12796fd thru 0x127b84a (0x214e bytes) 2024-09-13T07:13:07.842 INFO:teuthology.orchestra.run.smithi031.stdout:3291 read 0x68b8add thru 0x68b9d8a (0x12ae bytes) 2024-09-13T07:13:07.842 INFO:teuthology.orchestra.run.smithi031.stdout:3292 write 0x3b4aa19 thru 0x3b55e02 (0xb3ea bytes) 2024-09-13T07:13:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:3293 read 0x94c3e24 thru 0x94c7fc6 (0x41a3 bytes) 2024-09-13T07:13:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:3294 write 0x42603b8 thru 0x42698b4 (0x94fd bytes) 2024-09-13T07:13:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:3296 punch from 0x5a0400a to 0x5a059ae, (0x19a4 bytes) 2024-09-13T07:13:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:3297 write 0x59fcfd9 thru 0x5a01431 (0x4459 bytes) 2024-09-13T07:13:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:3298 writesame 0x1402eae thru 0x140d90e data_size 0x5d(0xaa61 bytes) 2024-09-13T07:13:07.886 INFO:teuthology.orchestra.run.smithi031.stdout:3299 trunc from 0xe2ef2c5 to 0x3f0d12f 2024-09-13T07:13:07.908 INFO:teuthology.orchestra.run.smithi031.stdout:3300 read 0x27918ac thru 0x2794961 (0x30b6 bytes) 2024-09-13T07:13:07.915 INFO:teuthology.orchestra.run.smithi031.stdout:3301 trunc from 0x3f0d12f to 0x953b4a8 2024-09-13T07:13:07.920 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:07.920 INFO:teuthology.orchestra.run.smithi031.stdout:3303 read 0x61e766b thru 0x61f1cf5 (0xa68b bytes) 2024-09-13T07:13:07.920 INFO:teuthology.orchestra.run.smithi031.stdout:3304 read 0x714347e thru 0x7152b31 (0xf6b4 bytes) 2024-09-13T07:13:07.926 INFO:teuthology.orchestra.run.smithi031.stdout:3307 trunc from 0x953b4a8 to 0x4d05c9a 2024-09-13T07:13:07.931 INFO:teuthology.orchestra.run.smithi031.stdout:3309 read 0x472f66f thru 0x4730ee8 (0x187a bytes) 2024-09-13T07:13:07.936 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:07.937+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:07.939 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:07.939 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:07.943 INFO:teuthology.orchestra.run.smithi031.stdout:3310 write 0xa38a7f8 thru 0xa3905ce (0x5dd7 bytes) 2024-09-13T07:13:07.947 INFO:teuthology.orchestra.run.smithi031.stdout:3312 writesame 0x3cd64d thru 0x3cedf8 data_size 0x1e(0x17ac bytes) 2024-09-13T07:13:07.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:07.953 INFO:teuthology.orchestra.run.smithi031.stdout:3315 write 0x61d1d82 thru 0x61d56df (0x395e bytes) 2024-09-13T07:13:07.964 INFO:teuthology.orchestra.run.smithi031.stdout:3317 writesame 0xb324282 thru 0xb324a4b data_size 0x1(0x7ca bytes) 2024-09-13T07:13:07.964 INFO:teuthology.orchestra.run.smithi031.stdout:3318 read 0xa3e74da thru 0xa3f513b (0xdc62 bytes) 2024-09-13T07:13:07.972 INFO:teuthology.orchestra.run.smithi031.stdout:3319 writesame 0xe8b54a8 thru 0xe8ba6fb data_size 0x16(0x5254 bytes) 2024-09-13T07:13:07.972 INFO:teuthology.orchestra.run.smithi031.stdout:3322 write 0x78ca1c6 thru 0x78cd12a (0x2f65 bytes) 2024-09-13T07:13:07.975 INFO:teuthology.orchestra.run.smithi031.stdout:3324 write 0x8feaa2 thru 0x905e29 (0x7388 bytes) 2024-09-13T07:13:07.989 INFO:teuthology.orchestra.run.smithi031.stdout:3325 write 0x42777a thru 0x42f970 (0x81f7 bytes) 2024-09-13T07:13:07.999 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:07.999 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:07.999 INFO:teuthology.orchestra.run.smithi031.stdout:3329 write 0xe3eb65d thru 0xe3ecfc9 (0x196d bytes) 2024-09-13T07:13:08.002 INFO:teuthology.orchestra.run.smithi031.stdout:3331 read 0x9b713be thru 0x9b7838f (0x6fd2 bytes) 2024-09-13T07:13:08.002 INFO:teuthology.orchestra.run.smithi031.stdout:3334 punch from 0x83f94c7 to 0x83feec2, (0x59fb bytes) 2024-09-13T07:13:08.010 INFO:teuthology.orchestra.run.smithi031.stdout:3335 trunc from 0xe8ba6fc to 0x801f1b2 2024-09-13T07:13:08.025 INFO:teuthology.orchestra.run.smithi031.stdout:3336 trunc from 0x801f1b2 to 0x2a56ef9 2024-09-13T07:13:08.037 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:08.038+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:08.087 INFO:teuthology.orchestra.run.smithi031.stdout:3337 write 0x1676b1c thru 0x16816a4 (0xab89 bytes) 2024-09-13T07:13:08.088 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:08.088 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:08.098 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:08.103 INFO:teuthology.orchestra.run.smithi031.stdout:3339 writesame 0x3273e87 thru 0x3276d73 data_size 0x1(0x2eed bytes) 2024-09-13T07:13:08.111 INFO:teuthology.orchestra.run.smithi031.stdout:3340 trunc from 0x3276d74 to 0x8ffe907 2024-09-13T07:13:08.116 INFO:teuthology.orchestra.run.smithi031.stdout:3341 punch from 0x1f19885 to 0x1f19c86, (0x401 bytes) 2024-09-13T07:13:08.116 INFO:teuthology.orchestra.run.smithi031.stdout:3342 read 0x2ff8f3b thru 0x2fff1a8 (0x626e bytes) 2024-09-13T07:13:08.116 INFO:teuthology.orchestra.run.smithi031.stdout:3343 writesame 0x806f1d8 thru 0x807b727 data_size 0x70(0xc550 bytes) 2024-09-13T07:13:08.121 INFO:teuthology.orchestra.run.smithi031.stdout:3344 write 0x7f0cfce thru 0x7f170b3 (0xa0e6 bytes) 2024-09-13T07:13:08.124 INFO:teuthology.orchestra.run.smithi031.stdout:3346 write 0x55229de thru 0x552b152 (0x8775 bytes) 2024-09-13T07:13:08.128 INFO:teuthology.orchestra.run.smithi031.stdout:3347 read 0x71ab711 thru 0x71b4551 (0x8e41 bytes) 2024-09-13T07:13:08.128 INFO:teuthology.orchestra.run.smithi031.stdout:3349 punch from 0x63df4b2 to 0x63e334b, (0x3e99 bytes) 2024-09-13T07:13:08.129 INFO:teuthology.orchestra.run.smithi031.stdout:3351 trunc from 0x8ffe907 to 0x83e2d28 2024-09-13T07:13:08.137 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:08.138+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:08.138 INFO:teuthology.orchestra.run.smithi031.stdout:3352 read 0x62710ed thru 0x627bcd9 (0xabed bytes) 2024-09-13T07:13:08.139 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.146 INFO:teuthology.orchestra.run.smithi031.stdout:3354 trunc from 0x83e2d28 to 0xe9141a8 2024-09-13T07:13:08.151 INFO:teuthology.orchestra.run.smithi031.stdout:3355 write 0x25214e7 thru 0x2524e9b (0x39b5 bytes) 2024-09-13T07:13:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:3358 trunc from 0xe9141a8 to 0x5bd551e 2024-09-13T07:13:08.183 INFO:teuthology.orchestra.run.smithi031.stdout:3359 trunc from 0x5bd551e to 0x1f23dcc 2024-09-13T07:13:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:08.238 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:08.239+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:08.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:08.249 INFO:teuthology.orchestra.run.smithi031.stdout:3361 read 0x138e978 thru 0x1393670 (0x4cf9 bytes) 2024-09-13T07:13:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:3363 writesame 0x7acaea4 thru 0x7acd5a1 data_size 0x2e(0x26fe bytes) 2024-09-13T07:13:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:3364 read 0x403f229 thru 0x40491b1 (0x9f89 bytes) 2024-09-13T07:13:08.263 INFO:teuthology.orchestra.run.smithi031.stdout:3365 trunc from 0x7acd5a2 to 0x93fae8b 2024-09-13T07:13:08.267 INFO:teuthology.orchestra.run.smithi031.stdout:3366 read 0x838f5bf thru 0x838f9f2 (0x434 bytes) 2024-09-13T07:13:08.267 INFO:teuthology.orchestra.run.smithi031.stdout:3367 punch from 0x585f1f9 to 0x5866a27, (0x782e bytes) 2024-09-13T07:13:08.268 INFO:teuthology.orchestra.run.smithi031.stdout:3369 write 0x12aea60 thru 0x12bc776 (0xdd17 bytes) 2024-09-13T07:13:08.273 INFO:teuthology.orchestra.run.smithi031.stdout:3370 read 0x3961b6e thru 0x396db45 (0xbfd8 bytes) 2024-09-13T07:13:08.273 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.274 INFO:teuthology.orchestra.run.smithi031.stdout:3375 write 0x400305b thru 0x4012859 (0xf7ff bytes) 2024-09-13T07:13:08.289 INFO:teuthology.orchestra.run.smithi031.stdout:3376 trunc from 0x93fae8b to 0x73e981 2024-09-13T07:13:08.296 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:08.296+0000 7efee3c12640 -1 Falling back to public interface 2024-09-13T07:13:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:13:08.339 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:08.339+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:08.345 INFO:teuthology.orchestra.run.smithi031.stdout:3377 writesame 0xb8a94c thru 0xb9742b data_size 0x30(0xcae0 bytes) 2024-09-13T07:13:08.347 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.347 INFO:teuthology.orchestra.run.smithi031.stdout:3379 read 0x3b6f02 thru 0x3bfc2a (0x8d29 bytes) 2024-09-13T07:13:08.347 INFO:teuthology.orchestra.run.smithi031.stdout:3380 write 0x1e3118 thru 0x1f2570 (0xf459 bytes) 2024-09-13T07:13:08.349 INFO:teuthology.orchestra.run.smithi031.stdout:3381 punch from 0x72e62c to 0x73d457, (0xee2b bytes) 2024-09-13T07:13:08.349 INFO:teuthology.orchestra.run.smithi031.stdout:3382 punch from 0x2ff8a0 to 0x307412, (0x7b72 bytes) 2024-09-13T07:13:08.350 INFO:teuthology.orchestra.run.smithi031.stdout:3383 trunc from 0xb9742c to 0x1bad7f8 2024-09-13T07:13:08.355 INFO:teuthology.orchestra.run.smithi031.stdout:3385 read 0xef9d12 thru 0xefdad7 (0x3dc6 bytes) 2024-09-13T07:13:08.355 INFO:teuthology.orchestra.run.smithi031.stdout:3386 read 0x16801b8 thru 0x1683f4d (0x3d96 bytes) 2024-09-13T07:13:08.355 INFO:teuthology.orchestra.run.smithi031.stdout:3387 write 0x25ef7 thru 0x2a59e (0x46a8 bytes) 2024-09-13T07:13:08.363 INFO:teuthology.orchestra.run.smithi031.stdout:3388 writesame 0x2a45838 thru 0x2a4ebe0 data_size 0x1(0x93a9 bytes) 2024-09-13T07:13:08.363 INFO:teuthology.orchestra.run.smithi031.stdout:3389 trunc from 0x2a4ebe1 to 0x1fba6c7 2024-09-13T07:13:08.368 INFO:teuthology.orchestra.run.smithi031.stdout:3390 read 0x1b51fb8 thru 0x1b57ca4 (0x5ced bytes) 2024-09-13T07:13:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:3391 writesame 0xb1f04b3 thru 0xb2002ef data_size 0x5(0xfe3d bytes) 2024-09-13T07:13:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:3393 write 0x6e02507 thru 0x6e073b3 (0x4ead bytes) 2024-09-13T07:13:08.392 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:08.392 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:08.399 INFO:teuthology.orchestra.run.smithi031.stdout:3395 trunc from 0xb2002f0 to 0x1228e1f 2024-09-13T07:13:08.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:08.407 INFO:teuthology.orchestra.run.smithi031.stdout:3396 read 0x14e64c thru 0x15ddce (0xf783 bytes) 2024-09-13T07:13:08.407 INFO:teuthology.orchestra.run.smithi031.stdout:3397 punch from 0x9e1e15 to 0x9e55b8, (0x37a3 bytes) 2024-09-13T07:13:08.408 INFO:teuthology.orchestra.run.smithi031.stdout:3398 read 0x957f91 thru 0x95f75f (0x77cf bytes) 2024-09-13T07:13:08.409 INFO:teuthology.orchestra.run.smithi031.stdout:3400 punch from 0x51ce41 to 0x524e56, (0x8015 bytes) 2024-09-13T07:13:08.417 INFO:teuthology.orchestra.run.smithi031.stdout:3401 write 0x4821be3 thru 0x482fd11 (0xe12f bytes) 2024-09-13T07:13:08.422 INFO:teuthology.orchestra.run.smithi031.stdout:3403 read 0x43b9cc2 thru 0x43c4121 (0xa460 bytes) 2024-09-13T07:13:08.422 INFO:teuthology.orchestra.run.smithi031.stdout:3405 punch from 0x15d4856 to 0x15e02a2, (0xba4c bytes) 2024-09-13T07:13:08.430 INFO:teuthology.orchestra.run.smithi031.stdout:3407 write 0x68623cb thru 0x6866098 (0x3cce bytes) 2024-09-13T07:13:08.433 INFO:teuthology.orchestra.run.smithi031.stdout:3408 write 0x63c4662 thru 0x63d0647 (0xbfe6 bytes) 2024-09-13T07:13:08.437 INFO:teuthology.orchestra.run.smithi031.stdout:3409 write 0x4817934 thru 0x48258a0 (0xdf6d bytes) 2024-09-13T07:13:08.440 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:08.440+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:08.442 INFO:teuthology.orchestra.run.smithi031.stdout:3410 writesame 0x6693288 thru 0x66a2702 data_size 0x1(0xf47b bytes) 2024-09-13T07:13:08.459 INFO:teuthology.orchestra.run.smithi031.stdout:3411 writesame 0x1bd931 thru 0x1c292d data_size 0x1(0x4ffd bytes) 2024-09-13T07:13:08.464 INFO:teuthology.orchestra.run.smithi031.stdout:3412 read 0xda4b45 thru 0xdad0b4 (0x8570 bytes) 2024-09-13T07:13:08.465 INFO:teuthology.orchestra.run.smithi031.stdout:3414 write 0x214c10c thru 0x21577d0 (0xb6c5 bytes) 2024-09-13T07:13:08.471 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.471 INFO:teuthology.orchestra.run.smithi031.stdout:3416 punch from 0x613c456 to 0x613e312, (0x1ebc bytes) 2024-09-13T07:13:08.471 INFO:teuthology.orchestra.run.smithi031.stdout:3417 read 0xe8fdeb thru 0xe91a1a (0x1c30 bytes) 2024-09-13T07:13:08.478 INFO:teuthology.orchestra.run.smithi031.stdout:3419 trunc from 0x6866099 to 0x96d013 2024-09-13T07:13:08.500 INFO:teuthology.orchestra.run.smithi031.stdout:3420 writesame 0x3e08910 thru 0x3e1501f data_size 0x8(0xc710 bytes) 2024-09-13T07:13:08.501 INFO:teuthology.orchestra.run.smithi031.stdout:3421 punch from 0x34df665 to 0x34e151a, (0x1eb5 bytes) 2024-09-13T07:13:08.507 INFO:teuthology.orchestra.run.smithi031.stdout:3422 write 0x5af92cb thru 0x5afc191 (0x2ec7 bytes) 2024-09-13T07:13:08.511 INFO:teuthology.orchestra.run.smithi031.stdout:3423 read 0x33f1af9 thru 0x33f1b5e (0x66 bytes) 2024-09-13T07:13:08.511 INFO:teuthology.orchestra.run.smithi031.stdout:3426 punch from 0x3c1b85a to 0x3c24024, (0x87ca bytes) 2024-09-13T07:13:08.511 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.511 INFO:teuthology.orchestra.run.smithi031.stdout:3429 writesame 0x36cb556 thru 0x36d4b84 data_size 0x1(0x962f bytes) 2024-09-13T07:13:08.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:08.528 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:08.529 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:08.535 INFO:teuthology.orchestra.run.smithi031.stdout:3430 trunc from 0x5afc192 to 0xe1b2589 2024-09-13T07:13:08.540 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.541 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:08.542+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:08.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:08.547 INFO:teuthology.orchestra.run.smithi031.stdout:3432 write 0xed24641 thru 0xed284aa (0x3e6a bytes) 2024-09-13T07:13:08.549 INFO:teuthology.orchestra.run.smithi031.stdout:3433 read 0xd24edd0 thru 0xd257cf7 (0x8f28 bytes) 2024-09-13T07:13:08.550 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.550 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.550 INFO:teuthology.orchestra.run.smithi031.stdout:3436 writesame 0x5530f70 thru 0x553f247 data_size 0x22(0xe2d8 bytes) 2024-09-13T07:13:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:3443 writesame 0xa130309 thru 0xa133099 data_size 0x1(0x2d91 bytes) 2024-09-13T07:13:08.562 INFO:teuthology.orchestra.run.smithi031.stdout:3445 punch from 0x3f9d9f2 to 0x3fa56f5, (0x7d03 bytes) 2024-09-13T07:13:08.562 INFO:teuthology.orchestra.run.smithi031.stdout:3446 writesame 0x5d502da thru 0x5d54518 data_size 0x1(0x423f bytes) 2024-09-13T07:13:08.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.568 INFO:teuthology.orchestra.run.smithi031.stdout:3449 write 0x93c3805 thru 0x93c6d70 (0x356c bytes) 2024-09-13T07:13:08.588 INFO:teuthology.orchestra.run.smithi031.stdout:3450 trunc from 0xed284ab to 0xba9218 2024-09-13T07:13:08.602 INFO:teuthology.orchestra.run.smithi031.stdout:3451 punch from 0x38f24d to 0x396c1b, (0x79ce bytes) 2024-09-13T07:13:08.614 INFO:teuthology.orchestra.run.smithi031.stdout:3452 trunc from 0xba9218 to 0x9a098e2 2024-09-13T07:13:08.626 INFO:teuthology.orchestra.run.smithi031.stdout:3455 trunc from 0x9a098e2 to 0x48d037c 2024-09-13T07:13:08.640 INFO:teuthology.orchestra.run.smithi031.stdout:3457 writesame 0x724db8a thru 0x7250ecb data_size 0x36(0x3342 bytes) 2024-09-13T07:13:08.640 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.640 INFO:teuthology.orchestra.run.smithi031.stdout:3460 write 0x2d1fdaf thru 0x2d2eafc (0xed4e bytes) 2024-09-13T07:13:08.642 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:08.643+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:08.645 INFO:teuthology.orchestra.run.smithi031.stdout:3462 read 0x61d534d thru 0x61db65b (0x630f bytes) 2024-09-13T07:13:08.645 INFO:teuthology.orchestra.run.smithi031.stdout:3463 read 0x4b023f2 thru 0x4b02656 (0x265 bytes) 2024-09-13T07:13:08.645 INFO:teuthology.orchestra.run.smithi031.stdout:3464 read 0x70e21d8 thru 0x70eba90 (0x98b9 bytes) 2024-09-13T07:13:08.645 INFO:teuthology.orchestra.run.smithi031.stdout:3465 punch from 0x123edfa to 0x124e86b, (0xfa71 bytes) 2024-09-13T07:13:08.645 INFO:teuthology.orchestra.run.smithi031.stdout:3466 read 0x14d0128 thru 0x14d7718 (0x75f1 bytes) 2024-09-13T07:13:08.646 INFO:teuthology.orchestra.run.smithi031.stdout:3467 read 0x36f1e39 thru 0x36f240b (0x5d3 bytes) 2024-09-13T07:13:08.646 INFO:teuthology.orchestra.run.smithi031.stdout:3468 read 0x3974bab thru 0x397a922 (0x5d78 bytes) 2024-09-13T07:13:08.646 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.652 INFO:teuthology.orchestra.run.smithi031.stdout:3470 trunc from 0x7250ecc to 0xe5d9b0 2024-09-13T07:13:08.663 INFO:teuthology.orchestra.run.smithi031.stdout:3473 trunc from 0xe5d9b0 to 0x1645dbb 2024-09-13T07:13:08.667 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.668 INFO:teuthology.orchestra.run.smithi031.stdout:3476 read 0x13b0022 thru 0x13bc7d1 (0xc7b0 bytes) 2024-09-13T07:13:08.668 INFO:teuthology.orchestra.run.smithi031.stdout:3477 punch from 0x14e7f7d to 0x14ee532, (0x65b5 bytes) 2024-09-13T07:13:08.668 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.668 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:08.668 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:08.669 INFO:teuthology.orchestra.run.smithi031.stdout:3479 write 0x9cddd6 thru 0x9d97ce (0xb9f9 bytes) 2024-09-13T07:13:08.676 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:08.676 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:08.677 INFO:teuthology.orchestra.run.smithi031.stdout:3480 write 0x3649451 thru 0x3649716 (0x2c6 bytes) 2024-09-13T07:13:08.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:08.689 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:08.695 INFO:teuthology.orchestra.run.smithi031.stdout:3482 writesame 0xc6409bb thru 0xc64761b data_size 0x1f(0x6c61 bytes) 2024-09-13T07:13:08.695 INFO:teuthology.orchestra.run.smithi031.stdout:3484 writesame 0x8f91ab4 thru 0x8f97c65 data_size 0x52(0x61b2 bytes) 2024-09-13T07:13:08.701 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.701 INFO:teuthology.orchestra.run.smithi031.stdout:3486 write 0x8b9ddbe thru 0x8ba2a60 (0x4ca3 bytes) 2024-09-13T07:13:08.706 INFO:teuthology.orchestra.run.smithi031.stdout:3487 write 0x9b5cf3f thru 0x9b65952 (0x8a14 bytes) 2024-09-13T07:13:08.708 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:08.708 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:08.712 INFO:teuthology.orchestra.run.smithi031.stdout:3488 read 0x31aa3ae thru 0x31b4a35 (0xa688 bytes) 2024-09-13T07:13:08.716 INFO:teuthology.orchestra.run.smithi031.stdout:3489 trunc from 0xc64761c to 0x98d852c 2024-09-13T07:13:08.727 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:08.733 INFO:teuthology.orchestra.run.smithi031.stdout:3490 writesame 0xa990fbf thru 0xa993c8a data_size 0x3d(0x2ccc bytes) 2024-09-13T07:13:08.743 INFO:teuthology.orchestra.run.smithi031.stdout:3491 write 0xe30dedc thru 0xe31db73 (0xfc98 bytes) 2024-09-13T07:13:08.743 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:08.744+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:08.746 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:08.746 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:08.753 INFO:teuthology.orchestra.run.smithi031.stdout:3492 read 0xadf52d9 thru 0xae03131 (0xde59 bytes) 2024-09-13T07:13:08.753 INFO:teuthology.orchestra.run.smithi031.stdout:3493 read 0x56090d6 thru 0x561110a (0x8035 bytes) 2024-09-13T07:13:08.753 INFO:teuthology.orchestra.run.smithi031.stdout:3494 punch from 0x84aa430 to 0x84b2e9b, (0x8a6b bytes) 2024-09-13T07:13:08.753 INFO:teuthology.orchestra.run.smithi031.stdout:3495 write 0x69d8423 thru 0x69e0f4b (0x8b29 bytes) 2024-09-13T07:13:08.763 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:08.763 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:08.764 INFO:teuthology.orchestra.run.smithi031.stdout:3497 read 0x273bf2b thru 0x274801d (0xc0f3 bytes) 2024-09-13T07:13:08.768 INFO:teuthology.orchestra.run.smithi031.stdout:3498 trunc from 0xe31db74 to 0xa9ce194 2024-09-13T07:13:08.782 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:13:08.784 INFO:teuthology.orchestra.run.smithi031.stdout:3502 punch from 0x20c672c to 0x20cc284, (0x5b58 bytes) 2024-09-13T07:13:08.784 INFO:teuthology.orchestra.run.smithi031.stdout:3503 read 0x8cf11bd thru 0x8cf23f1 (0x1235 bytes) 2024-09-13T07:13:08.784 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.784 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:08.785 INFO:teuthology.orchestra.run.smithi031.stdout:3507 clone 21 order 23 su 65536 sc 4 2024-09-13T07:13:08.785 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:13:08.786 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.516102750000002, 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:08.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:08.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.514624725000001, 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:08.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.822 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.511807676, 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:08.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.942378716, 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.831 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.929755119999999, 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:13:08.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:13:08.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.186173352000001, 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:13:08.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.185636124999998, 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:13:08.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:08.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.046554488999998, 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4162 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'113, mlcod=29'110)", 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.999746771000002, 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028573499999999998, 2024-09-13T07:13:08.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814277+0000", 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814283+0000", 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814293+0000", 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0299000000000001e-05 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814298+0000", 2024-09-13T07:13:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:13:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814345+0000", 2024-09-13T07:13:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7707999999999998e-05 2024-09-13T07:13:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814365+0000", 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9836000000000001e-05 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817100+0000", 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027346969999999999 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817135+0000", 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5194999999999998e-05 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.999207196, 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.971723091000001, 2024-09-13T07:13:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:13:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4195 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'122, mlcod=29'119)", 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.966187483999999, 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027697870000000001, 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847837+0000", 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847836+0000", 2024-09-13T07:13:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847841+0000", 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847850+0000", 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2390000000000002e-06 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847854+0000", 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847880+0000", 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.847900+0000", 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0015000000000001e-05 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850568+0000", 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002668283 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.850607+0000", 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8349999999999997e-05 2024-09-13T07:13:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.965708667000001, 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.914016372999999, 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:13:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.377792525, 2024-09-13T07:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7730 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'61, mlcod=34'54)", 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.376890746000001, 2024-09-13T07:13:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028499979999999999, 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437133+0000", 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437139+0000", 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7409999999999998e-06 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437151+0000", 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1946e-05 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437157+0000", 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437201+0000", 2024-09-13T07:13:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3917e-05 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437221+0000", 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439958+0000", 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027367300000000002 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439984+0000", 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5551999999999999e-05 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8060 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'136, mlcod=34'133)", 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.050537963, 2024-09-13T07:13:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002785984, 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763487+0000", 2024-09-13T07:13:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763486+0000", 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763492+0000", 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5050000000000004e-06 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763510+0000", 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7833e-05 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763517+0000", 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763556+0000", 2024-09-13T07:13:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8702999999999999e-05 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.763577+0000", 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1375000000000001e-05 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766246+0000", 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026686890000000001 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.766273+0000", 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6930000000000001e-05 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.617639347000001, 2024-09-13T07:13:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:13:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1072306580000006, 2024-09-13T07:13:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:64537 3.9 e47/19 3:97a74db9:::rbd_data.117332306d66.0000000000000002:head v 47'372, mlcod=45'370)", 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1946172260000001, 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092884539999999998, 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.624988+0000", 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055812279999999997 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625018+0000", 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9496999999999999e-05 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625031+0000", 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625078+0000", 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7290000000000003e-05 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625096+0000", 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8091999999999999e-05 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628662+0000", 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035652449999999999 2024-09-13T07:13:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628696+0000", 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4035999999999997e-05 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:08.913 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:08.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:08.916 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:08.916+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:09.016 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:09.017+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:09.063 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:09.064 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:09.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:09.116 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:09.117+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:09.179 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:09.179 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:09.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:09.218 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:09.219+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:09.319 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:09.320+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:09.337 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:09.337 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7754707060000001, 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:13:09.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.775445392, 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7754110260000004, 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7753822020000003, 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7753639049999999, 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:13:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.775339099, 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.775321226, 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7752948850000001, 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:13:09.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7752681629999998, 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7752505779999996, 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:13:09.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:09.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7752239400000001, 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:13:09.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:13:09.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7752064799999996, 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:13:09.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7751795220000002, 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:13:09.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:13:09.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7751533180000001, 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:13:09.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.775135326, 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:13:09.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7750435590000002, 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:13:09.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7750212589999999, 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7749902579999999, 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7749655019999997, 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:09.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7749478100000005, 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:09.424 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:09.425 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:09.426 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone20 from 0xa9ce194 (overlap 0x73e981) to 0x512373 2024-09-13T07:13:09.426 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:09.426+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:09.526 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T07:13:09.526 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:09.527+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:09.533 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71720 3.f 3:f1787d65:::rbd_object_map.1173ab534c79:head [call lock.lock in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.534905+0000", 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00048996700000000001, 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00051532599999999998, 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.534 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71720 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.534905+0000", 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.534905+0000", 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.534904+0000", 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.534909+0000", 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9309999999999999e-06 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.534920+0000", 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0492000000000001e-05 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.536 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.534924+0000", 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7700000000000001e-06 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.534965+0000", 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0240000000000001e-05 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.535022+0000", 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.537 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.538 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-13T07:13:09.538 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 1 2024-09-13T07:13:09.538 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:09.539 INFO:teuthology.orchestra.run.smithi031.stdout:3509 writesame 0xce89263 thru 0xce8db87 data_size 0x23(0x4925 bytes) 2024-09-13T07:13:09.539 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:09.539 INFO:teuthology.orchestra.run.smithi031.stdout:3515 write 0xac3e817 thru 0xac47d58 (0x9542 bytes) 2024-09-13T07:13:09.542 INFO:teuthology.orchestra.run.smithi031.stdout:3516 write 0x9904f07 thru 0x990a1a6 (0x52a0 bytes) 2024-09-13T07:13:09.543 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:09.549 INFO:teuthology.orchestra.run.smithi031.stdout:3517 write 0x3c25427 thru 0x3c25aec (0x6c6 bytes) 2024-09-13T07:13:09.555 INFO:teuthology.orchestra.run.smithi031.stdout:3518 punch from 0xadc412e to 0xadce31d, (0xa1ef bytes) 2024-09-13T07:13:09.555 INFO:teuthology.orchestra.run.smithi031.stdout:3519 write 0xb5c33cc thru 0xb5ce7d4 (0xb409 bytes) 2024-09-13T07:13:09.556 INFO:teuthology.orchestra.run.smithi031.stdout:3520 read 0x28659f1 thru 0x286d32b (0x793b bytes) 2024-09-13T07:13:09.556 INFO:teuthology.orchestra.run.smithi031.stdout:3521 read 0x4eb4323 thru 0x4eb4d45 (0xa23 bytes) 2024-09-13T07:13:09.557 INFO:teuthology.orchestra.run.smithi031.stdout:3523 write 0x22c70db thru 0x22c8079 (0xf9f bytes) 2024-09-13T07:13:09.563 INFO:teuthology.orchestra.run.smithi031.stdout:3524 read 0x1e5dbaf thru 0x1e618c0 (0x3d12 bytes) 2024-09-13T07:13:09.563 INFO:teuthology.orchestra.run.smithi031.stdout:3526 read 0x732f0e9 thru 0x7338cba (0x9bd2 bytes) 2024-09-13T07:13:09.563 INFO:teuthology.orchestra.run.smithi031.stdout:3527 read 0xab8337a thru 0xab91842 (0xe4c9 bytes) 2024-09-13T07:13:09.564 INFO:teuthology.orchestra.run.smithi031.stdout:3528 read 0x292da88 thru 0x2930205 (0x277e bytes) 2024-09-13T07:13:09.564 INFO:teuthology.orchestra.run.smithi031.stdout:3529 write 0xbbcb71c thru 0xbbcf540 (0x3e25 bytes) 2024-09-13T07:13:09.565 INFO:teuthology.orchestra.run.smithi031.stdout:3531 read 0x21538ea thru 0x21572ff (0x3a16 bytes) 2024-09-13T07:13:09.565 INFO:teuthology.orchestra.run.smithi031.stdout:3532 read 0x652fb4d thru 0x6532709 (0x2bbd bytes) 2024-09-13T07:13:09.577 INFO:teuthology.orchestra.run.smithi031.stdout:3533 trunc from 0xce8db88 to 0x298d753 2024-09-13T07:13:09.627 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:09.628+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:09.633 INFO:teuthology.orchestra.run.smithi031.stdout:3534 trunc from 0x298d753 to 0x105f3af 2024-09-13T07:13:09.652 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:09.652 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:09.653 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:09.653 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:09.653 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:09.662 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:09.690 INFO:teuthology.orchestra.run.smithi031.stdout:3535 writesame 0x492d1e thru 0x49fab0 data_size 0x1(0xcd93 bytes) 2024-09-13T07:13:09.725 INFO:teuthology.orchestra.run.smithi031.stdout:3536 write 0x67cca34 thru 0x67d1825 (0x4df2 bytes) 2024-09-13T07:13:09.728 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:09.729+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:09.729 INFO:teuthology.orchestra.run.smithi031.stdout:3538 writesame 0x1c16f13 thru 0x1c2615d data_size 0x1(0xf24b bytes) 2024-09-13T07:13:09.751 INFO:teuthology.orchestra.run.smithi031.stdout:3540 writesame 0x1bc62c8 thru 0x1bd47e4 data_size 0x93(0xe51d bytes) 2024-09-13T07:13:09.759 INFO:teuthology.orchestra.run.smithi031.stdout:3541 trunc from 0x67d1826 to 0xbf9ddfc 2024-09-13T07:13:09.770 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.554505036999998, 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:09.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:13:09.776 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52442 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3985408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806891019999997, 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040126756, 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.777 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52442 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590093+0000", 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0799999999999996e-06 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590096+0000", 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2679999999999999e-06 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590099+0000", 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.481e-06 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630022+0000", 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039923472000000002 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630106+0000", 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4264999999999994e-05 2024-09-13T07:13:09.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630215+0000", 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108399 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52443 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806761909999999, 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040282897999999998, 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52443 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:09.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590102+0000", 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2130000000000001e-06 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590105+0000", 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.881e-06 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590107+0000", 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6510000000000001e-06 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630253+0000", 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040145713999999999 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630327+0000", 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3813000000000006e-05 2024-09-13T07:13:09.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630384+0000", 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6842999999999998e-05 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52444 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4034560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806669899999997, 2024-09-13T07:13:09.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040488365999999998, 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52444 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590111+0000", 2024-09-13T07:13:09.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.176e-06 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590114+0000", 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7889999999999999e-06 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590116+0000", 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.435e-06 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630413+0000", 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040296998000000001 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630493+0000", 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9610999999999994e-05 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630598+0000", 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000105566 2024-09-13T07:13:09.790 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52445 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4059136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806588979999999, 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040607740000000003, 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.791 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52445 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590119+0000", 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.288e-06 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590122+0000", 2024-09-13T07:13:09.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7030000000000002e-06 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590124+0000", 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4250000000000001e-06 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630628+0000", 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040503577999999998 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630671+0000", 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3458999999999998e-05 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630726+0000", 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4490000000000002e-05 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52446 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4083712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806507530000001, 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040820130000000003, 2024-09-13T07:13:09.795 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52446 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590127+0000", 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.271e-06 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590130+0000", 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7389999999999999e-06 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590132+0000", 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630757+0000", 2024-09-13T07:13:09.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040624614000000003 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630836+0000", 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8913999999999997e-05 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630946+0000", 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110398 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52447 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4108288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806424819999997, 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041064639, 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52447 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590134+0000", 2024-09-13T07:13:09.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590136+0000", 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.266e-06 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590138+0000", 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.869e-06 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590141+0000", 2024-09-13T07:13:09.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6699999999999998e-06 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630984+0000", 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040843276999999997 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631072+0000", 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7386000000000007e-05 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631199+0000", 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012740000000000001 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52448 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806338209999998, 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041275991999999997, 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.804 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52448 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:09.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:09.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:09.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590144+0000", 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2109999999999999e-06 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590147+0000", 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8820000000000001e-06 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590150+0000", 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631231+0000", 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041081307999999997 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631298+0000", 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7340000000000002e-05 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631419+0000", 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000120986 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52449 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4157440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1806251620000001, 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041487405999999998, 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.810 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52449 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590151+0000", 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590153+0000", 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590156+0000", 2024-09-13T07:13:09.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.807e-06 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590158+0000", 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.458e-06 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631453+0000", 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041294850000000001 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631534+0000", 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0908999999999994e-05 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631639+0000", 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000105099 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.814 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52450 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4182016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.1805994630000001, 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041663446999999999, 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52450 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.815 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590177+0000", 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590179+0000", 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.705e-06 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590182+0000", 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3689999999999998e-06 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590185+0000", 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631666+0000", 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041480994 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631741+0000", 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4666999999999995e-05 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631841+0000", 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100449 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53407 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3158016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0786444399999997, 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040141932999999998, 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.819 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53407 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692132+0000", 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692134+0000", 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692140+0000", 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8080000000000001e-06 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692145+0000", 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732076+0000", 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039931543 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732162+0000", 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5657999999999999e-05 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732275+0000", 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011256600000000001 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.823 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53408 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3182592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0786283890000004, 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040246709999999998, 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53408 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692148+0000", 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692150+0000", 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.877e-06 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692156+0000", 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.015e-06 2024-09-13T07:13:09.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692161+0000", 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.544e-06 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732301+0000", 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040140051000000003 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732342+0000", 2024-09-13T07:13:09.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1325999999999997e-05 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732395+0000", 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3229999999999997e-05 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53409 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3207168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0786031820000002, 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040446375, 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.828 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53409 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692173+0000", 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692180+0000", 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2620000000000004e-06 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692186+0000", 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.861e-06 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692191+0000", 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732426+0000", 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040234501999999998 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732513+0000", 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6991999999999997e-05 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732620+0000", 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107413 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53410 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3231744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0785816400000003, 2024-09-13T07:13:09.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040545431999999999, 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53410 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692197+0000", 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1280000000000002e-06 2024-09-13T07:13:09.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692203+0000", 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8080000000000001e-06 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692208+0000", 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2399999999999998e-06 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732647+0000", 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040438395000000002 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732686+0000", 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9573000000000002e-05 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732741+0000", 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4731000000000002e-05 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.836 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53421 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0767242499999998, 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040071946999999997, 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.837 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53421 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694052+0000", 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694058+0000", 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7679999999999997e-06 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694064+0000", 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2700000000000001e-06 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694069+0000", 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734115+0000", 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040045348000000001 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734125+0000", 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0151e-05 2024-09-13T07:13:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53425 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0766400679999997, 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040069779, 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:09.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53425 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694139+0000", 2024-09-13T07:13:09.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1330000000000002e-06 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694145+0000", 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694149+0000", 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5680000000000001e-06 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734197+0000", 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040048148999999998 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734207+0000", 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3670000000000003e-06 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53426 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694153+0000", 2024-09-13T07:13:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0766236119999997, 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040066536, 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53426 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694153+0000", 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694153+0000", 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694153+0000", 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694155+0000", 2024-09-13T07:13:09.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0109999999999999e-06 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694161+0000", 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8549999999999998e-06 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694165+0000", 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2339999999999996e-06 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734216+0000", 2024-09-13T07:13:09.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040050832000000001 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734220+0000", 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9709999999999998e-06 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53427 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.0766073609999998, 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040080377, 2024-09-13T07:13:09.849 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53427 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694169+0000", 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694172+0000", 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2359999999999999e-06 2024-09-13T07:13:09.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694177+0000", 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8150000000000002e-06 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694182+0000", 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6380000000000003e-06 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734242+0000", 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040059715000000003 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734250+0000", 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3429999999999996e-06 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:71808 3.a e50/19 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head v 50'527, mlcod=50'525)", 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:09.853 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.18639882699999999, 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041667003000000001, 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584377+0000", 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584385+0000", 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2949999999999994e-06 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584402+0000", 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6458999999999999e-05 2024-09-13T07:13:09.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584409+0000", 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584457+0000", 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8130000000000002e-05 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584475+0000", 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7677e-05 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626008+0000", 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041532931000000002 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626045+0000", 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7654999999999998e-05 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71817 3.f 3:f1787d65:::rbd_object_map.1173ab534c79:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.185506741, 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040930686000000001, 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71817 2024-09-13T07:13:09.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585269+0000", 2024-09-13T07:13:09.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585275+0000", 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.237e-06 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585284+0000", 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0529999999999996e-06 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585288+0000", 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5029999999999999e-06 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585318+0000", 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0045000000000001e-05 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585364+0000", 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5985000000000001e-05 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:09.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626068+0000", 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040703678 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626114+0000", 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6508999999999998e-05 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626201+0000", 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6655000000000003e-05 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.862 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.863 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.863 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.863 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:09.865 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:09.867 INFO:teuthology.orchestra.run.smithi031.stdout:3542 trunc from 0xbf9ddfc to 0x157d06e 2024-09-13T07:13:09.867 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3544 read 0x9e1981 thru 0x9f0d82 (0xf402 bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3545 trunc from 0x157d06e to 0x9f76de2 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3546 write 0x5aa9125 thru 0x5aae027 (0x4f03 bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3547 read 0x49eb806 thru 0x49f712c (0xb927 bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3549 read 0x137a89c thru 0x13872e8 (0xca4d bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3550 punch from 0x7a52b61 to 0x7a5bcf8, (0x9197 bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3551 read 0x70c6c89 thru 0x70d6c21 (0xff99 bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3552 punch from 0x68d246 to 0x68dcb4, (0xa6e bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3555 writesame 0xb2ba669 thru 0xb2ba899 data_size 0x3(0x231 bytes) 2024-09-13T07:13:09.868 INFO:teuthology.orchestra.run.smithi031.stdout:3556 write 0x9bfd3f0 thru 0x9bffc40 (0x2851 bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3558 write 0x27a0dca thru 0x27a1810 (0xa47 bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3559 write 0x8e40d5f thru 0x8e4a138 (0x93da bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3560 read 0x28f3bfe thru 0x28fc976 (0x8d79 bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3561 read 0x96b38e0 thru 0x96bb099 (0x77ba bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3563 write 0xc1fcf20 thru 0xc206ffe (0xa0df bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3564 write 0x84d0dda thru 0x84d7195 (0x63bc bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3565 read 0x6839ea thru 0x685f40 (0x2557 bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3567 write 0x7829a67 thru 0x7838088 (0xe622 bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3569 write 0x9fd2ec2 thru 0x9fe152c (0xe66b bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3570 read 0x429d4f9 thru 0x429e627 (0x112f bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3572 read 0x969e381 thru 0x96ade1e (0xfa9e bytes) 2024-09-13T07:13:09.869 INFO:teuthology.orchestra.run.smithi031.stdout:3573 punch from 0x5dbc4fe to 0x5dbcd66, (0x868 bytes) 2024-09-13T07:13:09.870 INFO:teuthology.orchestra.run.smithi031.stdout:3574 write 0xe03ea2d thru 0xe048468 (0x9a3c bytes) 2024-09-13T07:13:09.870 INFO:teuthology.orchestra.run.smithi031.stdout:3575 writesame 0x84e9948 thru 0x84f87ac data_size 0x3(0xee65 bytes) 2024-09-13T07:13:09.870 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:09.870 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:09.865+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:09.875 INFO:teuthology.orchestra.run.smithi031.stdout:3577 trunc from 0xe048469 to 0x2f05f85 2024-09-13T07:13:09.890 INFO:teuthology.orchestra.run.smithi031.stdout:3579 read 0x1e1e855 thru 0x1e2d2c9 (0xea75 bytes) 2024-09-13T07:13:09.891 INFO:teuthology.orchestra.run.smithi031.stdout:3580 punch from 0x2774710 to 0x27782c6, (0x3bb6 bytes) 2024-09-13T07:13:09.898 INFO:teuthology.orchestra.run.smithi031.stdout:3581 trunc from 0x2f05f85 to 0x95c3f24 2024-09-13T07:13:09.907 INFO:teuthology.orchestra.run.smithi031.stdout:3582 trunc from 0x95c3f24 to 0x5aa21a2 2024-09-13T07:13:09.918 INFO:teuthology.orchestra.run.smithi031.stdout:3583 trunc from 0x5aa21a2 to 0x11f212e 2024-09-13T07:13:09.931 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:09.945 INFO:teuthology.orchestra.run.smithi031.stdout:3586 write 0x8a0e91b thru 0x8a11e53 (0x3539 bytes) 2024-09-13T07:13:09.954 INFO:teuthology.orchestra.run.smithi031.stdout:3588 trunc from 0x8a11e54 to 0xdf4f229 2024-09-13T07:13:09.964 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:09.965+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:09.968 INFO:teuthology.orchestra.run.smithi031.stdout:3589 trunc from 0xdf4f229 to 0x644b8cd 2024-09-13T07:13:09.977 INFO:teuthology.orchestra.run.smithi031.stdout:3590 write 0x1973b0a thru 0x1976058 (0x254f bytes) 2024-09-13T07:13:09.977 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:72520 3.16 3:6b3d9844:::rbd_data.1173ab534c79.0000000000000003:head [stat,write 6634250~9551 in=9551b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.979177+0000", 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00016892099999999999, 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000189061, 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 72520 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.978 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979177+0000", 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979177+0000", 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979175+0000", 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979190+0000", 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4966e-05 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979205+0000", 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5440000000000001e-05 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979212+0000", 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8589999999999999e-06 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979263+0000", 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1073999999999999e-05 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.979353+0000", 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.9618999999999997e-05 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:09.981 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-13T07:13:09.982 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 1 2024-09-13T07:13:09.982 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:09.982 INFO:teuthology.orchestra.run.smithi031.stdout:3591 writesame 0x1b5c750 thru 0x1b6406c data_size 0xf(0x791d bytes) 2024-09-13T07:13:09.982 INFO:teuthology.orchestra.run.smithi031.stdout:3592 read 0x472c189 thru 0x4736c0b (0xaa83 bytes) 2024-09-13T07:13:09.982 INFO:teuthology.orchestra.run.smithi031.stdout:3593 trunc from 0x644b8cd to 0x60dc13b 2024-09-13T07:13:09.986 INFO:teuthology.orchestra.run.smithi031.stdout:3594 read 0x1de81e thru 0x1e030e (0x1af1 bytes) 2024-09-13T07:13:09.987 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:09.989 INFO:teuthology.orchestra.run.smithi031.stdout:3595 trunc from 0x60dc13b to 0x7f3b42c 2024-09-13T07:13:09.995 INFO:teuthology.orchestra.run.smithi031.stdout:3596 read 0x30d0d25 thru 0x30d2fe6 (0x22c2 bytes) 2024-09-13T07:13:10.003 INFO:teuthology.orchestra.run.smithi031.stdout:3597 writesame 0xae632d4 thru 0xae6fe66 data_size 0x7(0xcb93 bytes) 2024-09-13T07:13:10.003 INFO:teuthology.orchestra.run.smithi031.stdout:3598 write 0x72c1fe8 thru 0x72cea3d (0xca56 bytes) 2024-09-13T07:13:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:3599 writesame 0xd1fbd5c thru 0xd20412b data_size 0x4c(0x83d0 bytes) 2024-09-13T07:13:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:3602 trunc from 0xd20412c to 0x594a560 2024-09-13T07:13:10.032 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.032 INFO:teuthology.orchestra.run.smithi031.stdout:3604 read 0x5487361 thru 0x548d532 (0x61d2 bytes) 2024-09-13T07:13:10.032 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.032 INFO:teuthology.orchestra.run.smithi031.stdout:3607 punch from 0x18e3fe3 to 0x18ea17a, (0x6197 bytes) 2024-09-13T07:13:10.032 INFO:teuthology.orchestra.run.smithi031.stdout:3609 read 0xc22ee3 thru 0xc2429b (0x13b9 bytes) 2024-09-13T07:13:10.032 INFO:teuthology.orchestra.run.smithi031.stdout:3611 write 0x2dfbeb8 thru 0x2dffebc (0x4005 bytes) 2024-09-13T07:13:10.034 INFO:teuthology.orchestra.run.smithi031.stdout:3613 write 0x199a13d thru 0x199b689 (0x154d bytes) 2024-09-13T07:13:10.035 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.045 INFO:teuthology.orchestra.run.smithi031.stdout:3615 write 0x95d2de1 thru 0x95e19b4 (0xebd4 bytes) 2024-09-13T07:13:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:3616 write 0x4d90956 thru 0x4d97d2c (0x73d7 bytes) 2024-09-13T07:13:10.053 INFO:teuthology.orchestra.run.smithi031.stdout:3617 writesame 0x4827eff thru 0x482945b data_size 0x1(0x155d bytes) 2024-09-13T07:13:10.065 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:10.066+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:10.067 INFO:teuthology.orchestra.run.smithi031.stdout:3618 writesame 0xccb81a2 thru 0xccc28a7 data_size 0x1(0xa706 bytes) 2024-09-13T07:13:10.067 INFO:teuthology.orchestra.run.smithi031.stdout:3619 read 0xa8a4d34 thru 0xa8b0031 (0xb2fe bytes) 2024-09-13T07:13:10.069 INFO:teuthology.orchestra.run.smithi031.stdout:3623 trunc from 0xccc28a8 to 0xa6800c6 2024-09-13T07:13:10.076 INFO:teuthology.orchestra.run.smithi031.stdout:3625 writesame 0x529070 thru 0x535010 data_size 0x1(0xbfa1 bytes) 2024-09-13T07:13:10.090 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.097 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:10.097 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:10.098 INFO:teuthology.orchestra.run.smithi031.stdout:3628 write 0xd9239e4 thru 0xd93220b (0xe828 bytes) 2024-09-13T07:13:10.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:10.107 INFO:teuthology.orchestra.run.smithi031.stdout:3630 punch from 0xcd598bb to 0xcd6685b, (0xcfa0 bytes) 2024-09-13T07:13:10.107 INFO:teuthology.orchestra.run.smithi031.stdout:3631 writesame 0x575ff3a thru 0x5761207 data_size 0x1d(0x12ce bytes) 2024-09-13T07:13:10.109 INFO:teuthology.orchestra.run.smithi031.stdout:3632 read 0x6658069 thru 0x6665456 (0xd3ee bytes) 2024-09-13T07:13:10.109 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.109 INFO:teuthology.orchestra.run.smithi031.stdout:3634 writesame 0x867c286 thru 0x868a8cf data_size 0x7(0xe64a bytes) 2024-09-13T07:13:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:3635 write 0x566745b thru 0x566d290 (0x5e36 bytes) 2024-09-13T07:13:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:3636 punch from 0xc0ef931 to 0xc0f3cc3, (0x4392 bytes) 2024-09-13T07:13:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:3637 write 0x70992b8 thru 0x70a2864 (0x95ad bytes) 2024-09-13T07:13:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:3640 write 0x87817bf thru 0x87821f5 (0xa37 bytes) 2024-09-13T07:13:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:3641 write 0x6624889 thru 0x6632e8b (0xe603 bytes) 2024-09-13T07:13:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:3642 read 0x549c1d4 thru 0x54a1052 (0x4e7f bytes) 2024-09-13T07:13:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:3644 read 0xd008630 thru 0xd017897 (0xf268 bytes) 2024-09-13T07:13:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:3645 write 0x709dbe thru 0x711a09 (0x7c4c bytes) 2024-09-13T07:13:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:3649 read 0x4c435ec thru 0x4c47eb2 (0x48c7 bytes) 2024-09-13T07:13:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:3651 punch from 0xd0dd829 to 0xd0e9cc0, (0xc497 bytes) 2024-09-13T07:13:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:3653 write 0x9950c6c thru 0x9958d60 (0x80f5 bytes) 2024-09-13T07:13:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:3654 read 0xc889e2e thru 0xc88d5dd (0x37b0 bytes) 2024-09-13T07:13:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:3655 read 0xacbd5e1 thru 0xacbf54e (0x1f6e bytes) 2024-09-13T07:13:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:3657 write 0x4af00ff thru 0x4afe75d (0xe65f bytes) 2024-09-13T07:13:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:3658 writesame 0xa541521 thru 0xa54516d data_size 0x1(0x3c4d bytes) 2024-09-13T07:13:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:3659 writesame 0x1108fd3 thru 0x110eb13 data_size 0xd(0x5b41 bytes) 2024-09-13T07:13:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:3661 writesame 0xec27130 thru 0xec32dee data_size 0x1(0xbcbf bytes) 2024-09-13T07:13:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:3662 write 0x3b85f58 thru 0x3b929aa (0xca53 bytes) 2024-09-13T07:13:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:3664 write 0x40143b3 thru 0x401a983 (0x65d1 bytes) 2024-09-13T07:13:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:3665 write 0x8f6292e thru 0x8f65f4a (0x361d bytes) 2024-09-13T07:13:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:3666 punch from 0x6e67788 to 0x6e6e601, (0x6e79 bytes) 2024-09-13T07:13:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:3667 writesame 0xda71a8 thru 0xdb00c4 data_size 0x1(0x8f1d bytes) 2024-09-13T07:13:10.188 INFO:teuthology.orchestra.run.smithi031.stdout:3668 writesame 0x8be9492 thru 0x8bf5131 data_size 0x30(0xbca0 bytes) 2024-09-13T07:13:10.190 INFO:teuthology.orchestra.run.smithi031.stdout:3669 read 0xa345c96 thru 0xa345d20 (0x8b bytes) 2024-09-13T07:13:10.190 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:10.191+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:10.190 INFO:teuthology.orchestra.run.smithi031.stdout:3670 read 0x4cfbeca thru 0x4d0439f (0x84d6 bytes) 2024-09-13T07:13:10.191 INFO:teuthology.orchestra.run.smithi031.stdout:3671 write 0x3df4f9e thru 0x3df8b20 (0x3b83 bytes) 2024-09-13T07:13:10.204 INFO:teuthology.orchestra.run.smithi031.stdout:3672 trunc from 0xec32def to 0x38cfb9a 2024-09-13T07:13:10.217 INFO:teuthology.orchestra.run.smithi031.stdout:3674 writesame 0x302bc4c thru 0x302c765 data_size 0x1(0xb1a bytes) 2024-09-13T07:13:10.221 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.229 INFO:teuthology.orchestra.run.smithi031.stdout:3676 trunc from 0x38cfb9a to 0xa6d3afd 2024-09-13T07:13:10.234 INFO:teuthology.orchestra.run.smithi031.stdout:3677 read 0x14c211a thru 0x14c8e38 (0x6d1f bytes) 2024-09-13T07:13:10.239 INFO:teuthology.orchestra.run.smithi031.stdout:3678 trunc from 0xa6d3afd to 0xe8fddfc 2024-09-13T07:13:10.243 INFO:teuthology.orchestra.run.smithi031.stdout:3681 read 0xc6f6bf thru 0xc77a4e (0x8390 bytes) 2024-09-13T07:13:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:3682 write 0x5f784d8 thru 0x5f7f863 (0x738c bytes) 2024-09-13T07:13:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:3683 punch from 0x76f389c to 0x76fd98f, (0xa0f3 bytes) 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36784 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13295616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5927888560000003, 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028020702000000001, 2024-09-13T07:13:10.256 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36784 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663479+0000", 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1299999999999999e-06 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663485+0000", 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0430000000000004e-06 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663494+0000", 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6220000000000004e-06 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691423+0000", 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027928339 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691464+0000", 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1743999999999999e-05 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691498+0000", 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3235999999999998e-05 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36785 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13320192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5927672790000003, 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028091378, 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.261 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36785 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663498+0000", 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663500+0000", 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2400000000000002e-06 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663506+0000", 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663511+0000", 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6689999999999997e-06 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691518+0000", 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028007641999999999 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691557+0000", 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8139000000000002e-05 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691590+0000", 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3334000000000003e-05 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36786 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13344768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5927508899999996, 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028164842999999998, 2024-09-13T07:13:10.266 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36786 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663514+0000", 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663521+0000", 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663527+0000", 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8690000000000001e-06 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663532+0000", 2024-09-13T07:13:10.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4229999999999998e-06 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691609+0000", 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028077429000000001 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691647+0000", 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7540000000000003e-05 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691680+0000", 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3161000000000003e-05 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36787 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13369344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5927219360000002, 2024-09-13T07:13:10.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028232916, 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36787 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:10.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663546+0000", 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3070000000000001e-06 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663552+0000", 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1149999999999999e-06 2024-09-13T07:13:10.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663556+0000", 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691702+0000", 2024-09-13T07:13:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028145752999999999 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691741+0000", 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8611e-05 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691777+0000", 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5982999999999998e-05 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.276 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36788 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13393920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5927053969999996, 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028311981999999999, 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.277 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36788 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663565+0000", 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4519999999999996e-06 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663571+0000", 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5480000000000001e-06 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663576+0000", 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6079999999999998e-06 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691800+0000", 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028224468999999999 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691839+0000", 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8476999999999998e-05 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691872+0000", 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3868000000000002e-05 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36789 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13418496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5926862639999992, 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028396060000000001, 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36789 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663579+0000", 2024-09-13T07:13:10.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663581+0000", 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9369999999999998e-06 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663586+0000", 2024-09-13T07:13:10.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2800000000000003e-06 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663590+0000", 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.031e-06 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691896+0000", 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028305815000000002 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691941+0000", 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4956999999999997e-05 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691976+0000", 2024-09-13T07:13:10.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4422999999999999e-05 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36790 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13443072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5926713649999993, 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028477820000000001, 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.287 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36790 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663596+0000", 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2390000000000001e-06 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663602+0000", 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6740000000000002e-06 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663607+0000", 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5159999999999996e-06 2024-09-13T07:13:10.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691998+0000", 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028391937999999999 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692036+0000", 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7948999999999999e-05 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692072+0000", 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5893999999999999e-05 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36791 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13467648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5926554520000007, 2024-09-13T07:13:10.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028558888000000001, 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36791 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:10.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663612+0000", 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9420000000000002e-06 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663617+0000", 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.49e-06 2024-09-13T07:13:10.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663630+0000", 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2126e-05 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692094+0000", 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028464382 2024-09-13T07:13:10.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692134+0000", 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0234999999999997e-05 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692169+0000", 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5098999999999997e-05 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.297 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36792 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13492224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5926321160000008, 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028626243999999999, 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.298 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36792 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663633+0000", 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663636+0000", 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1909999999999999e-06 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663641+0000", 2024-09-13T07:13:10.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7089999999999999e-06 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663646+0000", 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8339999999999998e-06 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692191+0000", 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028544491000000002 2024-09-13T07:13:10.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692227+0000", 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6189999999999997e-05 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692260+0000", 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3198999999999998e-05 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36793 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13516800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5926157330000006, 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028699295999999999, 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36793 2024-09-13T07:13:10.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663656+0000", 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6220000000000003e-06 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663663+0000", 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1489999999999999e-06 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663667+0000", 2024-09-13T07:13:10.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5589999999999999e-06 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692280+0000", 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028612779000000001 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692316+0000", 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6492999999999998e-05 2024-09-13T07:13:10.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692349+0000", 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3052000000000001e-05 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:10.308 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5925938290000001, 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5925561780000006, 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:10.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:10.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5925369509999996, 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:10.319 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:10.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36799 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13664256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5913370560000004, 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027993130000000001, 2024-09-13T07:13:10.324 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36799 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664928+0000", 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664930+0000", 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0109999999999999e-06 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664936+0000", 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7810000000000002e-06 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664941+0000", 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5990000000000004e-06 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692841+0000", 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027900115999999999 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692887+0000", 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6063999999999998e-05 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692922+0000", 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4906000000000001e-05 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36800 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13688832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5913210969999998, 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028070522000000001, 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36800 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664944+0000", 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664950+0000", 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4949999999999999e-06 2024-09-13T07:13:10.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664956+0000", 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6829999999999996e-06 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664967+0000", 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1858000000000001e-05 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692944+0000", 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027976671000000002 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692981+0000", 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7311999999999998e-05 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693015+0000", 2024-09-13T07:13:10.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3856999999999998e-05 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36801 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13713408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5912940540000005, 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028136613000000001, 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36801 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664971+0000", 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664974+0000", 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4710000000000001e-06 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664980+0000", 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6620000000000002e-06 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664984+0000", 2024-09-13T07:13:10.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.561e-06 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693038+0000", 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028054374999999999 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693075+0000", 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6740999999999999e-05 2024-09-13T07:13:10.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693108+0000", 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3207000000000002e-05 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.340 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36802 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13737984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.341 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:10.341 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5912631659999992, 2024-09-13T07:13:10.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028199977000000001, 2024-09-13T07:13:10.341 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.341 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36802 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665002+0000", 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665005+0000", 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5790000000000002e-06 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665011+0000", 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7089999999999996e-06 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665022+0000", 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0631e-05 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693129+0000", 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028106822 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693168+0000", 2024-09-13T07:13:10.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9069999999999997e-05 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693203+0000", 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4826999999999997e-05 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36803 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13762560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5912393869999999, 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028271376000000001, 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36803 2024-09-13T07:13:10.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665028+0000", 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.294e-06 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665034+0000", 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5559999999999998e-06 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665038+0000", 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5179999999999998e-06 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693224+0000", 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028185387999999999 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693265+0000", 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0791999999999998e-05 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693298+0000", 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3198000000000003e-05 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36804 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13787136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:10.352 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.5912231139999999, 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028345479, 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36804 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665042+0000", 2024-09-13T07:13:10.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665044+0000", 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.199e-06 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665050+0000", 2024-09-13T07:13:10.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.733e-06 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665054+0000", 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2119999999999997e-06 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693319+0000", 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028264105000000001 2024-09-13T07:13:10.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693355+0000", 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6618000000000001e-05 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693388+0000", 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3065999999999998e-05 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.357 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.7164635390000003, 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.358 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:10.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:10.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:10.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.365 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.365 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.365 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:10.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:13:10.368 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:3684 trunc from 0xe8fddfc to 0x33c8a0e 2024-09-13T07:13:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:3685 writesame 0xa2d3722 thru 0xa2d7098 data_size 0x1(0x3977 bytes) 2024-09-13T07:13:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:3687 read 0x178e0d5 thru 0x1799612 (0xb53e bytes) 2024-09-13T07:13:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:3689 read 0x199965d thru 0x19a6c07 (0xd5ab bytes) 2024-09-13T07:13:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:3691 write 0x7cedf55 thru 0x7cfbdf2 (0xde9e bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3692 writesame 0xe4e6ef3 thru 0xe4ef25c data_size 0x7e(0x836a bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3694 read 0x139e638 thru 0x13abc09 (0xd5d2 bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3696 punch from 0xc565312 to 0xc56eff2, (0x9ce0 bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3700 write 0x2b37402 thru 0x2b441c4 (0xcdc3 bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3702 read 0xb0021bc thru 0xb0084c4 (0x6309 bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3704 writesame 0xba5fd3d thru 0xba6d36c data_size 0x2e(0xd630 bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3707 read 0x45b2ecd thru 0x45bb8d9 (0x8a0d bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3709 writesame 0x64edf00 thru 0x64f3bfe data_size 0x13(0x5cff bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3710 read 0x5e6a376 thru 0x5e74a3b (0xa6c6 bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3711 write 0x37e38e5 thru 0x37f3401 (0xfb1d bytes) 2024-09-13T07:13:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:3712 punch from 0x2bb99fa to 0x2bc1165, (0x776b bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3714 punch from 0xe4e4e72 to 0xe4eca6a, (0x7bf8 bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3715 writesame 0x1fdd17a thru 0x1fec772 data_size 0x1(0xf5f9 bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3716 read 0x6f219d5 thru 0x6f30455 (0xea81 bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3718 read 0x85363be thru 0x853b07b (0x4cbe bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3719 write 0xa287c3d thru 0xa290d6e (0x9132 bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3720 write 0xaac93eb thru 0xaad565b (0xc271 bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3721 punch from 0x72b2e5b to 0x72bd396, (0xa53b bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:3722 write 0x5753664 thru 0x5759857 (0x61f4 bytes) 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3729 punch from 0x71363e8 to 0x71401a8, (0x9dc0 bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3730 punch from 0xdfe3bff to 0xdfec43c, (0x883d bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3731 write 0x3f28778 thru 0x3f2d6cf (0x4f58 bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3732 read 0x3aa8bc0 thru 0x3ab12be (0x86ff bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3734 write 0xd2289b0 thru 0xd22c4f0 (0x3b41 bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3735 writesame 0xdca9f7a thru 0xdcabca7 data_size 0x1e(0x1d2e bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3737 writesame 0x7239afe thru 0x7243c68 data_size 0x1f(0xa16b bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3738 writesame 0xc898e68 thru 0xc89d58f data_size 0x48(0x4728 bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3739 write 0x59b6320 thru 0x59c0300 (0x9fe1 bytes) 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:3742 writesame 0x24d1c86 thru 0x24da94c data_size 0x29(0x8cc7 bytes) 2024-09-13T07:13:10.375 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:10.367+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:3745 trunc from 0xe4ef25d to 0x7ef5979 2024-09-13T07:13:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:3746 write 0x669a374 thru 0x66a724c (0xced9 bytes) 2024-09-13T07:13:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:3747 read 0x3c0d0f9 thru 0x3c18434 (0xb33c bytes) 2024-09-13T07:13:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:3748 read 0x74a2c83 thru 0x74a853c (0x58ba bytes) 2024-09-13T07:13:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:3749 read 0x4812d31 thru 0x481ec89 (0xbf59 bytes) 2024-09-13T07:13:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:3751 write 0xce9d882 thru 0xceaa8cc (0xd04b bytes) 2024-09-13T07:13:10.416 INFO:teuthology.orchestra.run.smithi031.stdout:3752 write 0xdb838e9 thru 0xdb8e503 (0xac1b bytes) 2024-09-13T07:13:10.419 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.420 INFO:teuthology.orchestra.run.smithi031.stdout:3755 punch from 0x4e8ca03 to 0x4e98b81, (0xc17e bytes) 2024-09-13T07:13:10.420 INFO:teuthology.orchestra.run.smithi031.stdout:3756 write 0x673cb45 thru 0x6741bb0 (0x506c bytes) 2024-09-13T07:13:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:3757 trunc from 0xdb8e504 to 0x4af2bf7 2024-09-13T07:13:10.443 INFO:teuthology.orchestra.run.smithi031.stdout:3758 write 0x3701673 thru 0x37029b3 (0x1341 bytes) 2024-09-13T07:13:10.444 INFO:teuthology.orchestra.run.smithi031.stdout:3759 write 0x39fdafa thru 0x3a05f62 (0x8469 bytes) 2024-09-13T07:13:10.445 INFO:teuthology.orchestra.run.smithi031.stdout:3761 read 0x48fd11b thru 0x490321d (0x6103 bytes) 2024-09-13T07:13:10.446 INFO:teuthology.orchestra.run.smithi031.stdout:3762 punch from 0x328b492 to 0x3297629, (0xc197 bytes) 2024-09-13T07:13:10.446 INFO:teuthology.orchestra.run.smithi031.stdout:3763 read 0x2f5baa6 thru 0x2f62ef7 (0x7452 bytes) 2024-09-13T07:13:10.456 INFO:teuthology.orchestra.run.smithi031.stdout:3764 write 0x7ff1cc2 thru 0x7ff660d (0x494c bytes) 2024-09-13T07:13:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:3766 punch from 0x2eb0ee1 to 0x2ebb4ba, (0xa5d9 bytes) 2024-09-13T07:13:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:3768 read 0x63cd2bd thru 0x63d9d31 (0xca75 bytes) 2024-09-13T07:13:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:3771 read 0x7cc36c4 thru 0x7cc7e6c (0x47a9 bytes) 2024-09-13T07:13:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:3772 write 0x13344d4 thru 0x133bb46 (0x7673 bytes) 2024-09-13T07:13:10.460 INFO:teuthology.orchestra.run.smithi031.stdout:3773 writesame 0x6474671 thru 0x6478226 data_size 0x1(0x3bb6 bytes) 2024-09-13T07:13:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:3775 writesame 0x4572310 thru 0x457dae1 data_size 0x5d(0xb7d2 bytes) 2024-09-13T07:13:10.467 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:10.467+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:10.468 INFO:teuthology.orchestra.run.smithi031.stdout:3776 punch from 0x16786d6 to 0x167a3c4, (0x1cee bytes) 2024-09-13T07:13:10.468 INFO:teuthology.orchestra.run.smithi031.stdout:3777 read 0x907c12 thru 0x91084d (0x8c3c bytes) 2024-09-13T07:13:10.468 INFO:teuthology.orchestra.run.smithi031.stdout:3778 read 0x78bf69d thru 0x78c17fa (0x215e bytes) 2024-09-13T07:13:10.469 INFO:teuthology.orchestra.run.smithi031.stdout:3779 read 0x42e37f5 thru 0x42e69ee (0x31fa bytes) 2024-09-13T07:13:10.469 INFO:teuthology.orchestra.run.smithi031.stdout:3781 read 0x36a7dd3 thru 0x36b3f37 (0xc165 bytes) 2024-09-13T07:13:10.469 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.469 INFO:teuthology.orchestra.run.smithi031.stdout:3783 read 0x493f60e thru 0x4948396 (0x8d89 bytes) 2024-09-13T07:13:10.470 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.470 INFO:teuthology.orchestra.run.smithi031.stdout:3785 read 0x55b8bf0 thru 0x55bffeb (0x73fc bytes) 2024-09-13T07:13:10.470 INFO:teuthology.orchestra.run.smithi031.stdout:3786 read 0x592e0b3 thru 0x5931591 (0x34df bytes) 2024-09-13T07:13:10.471 INFO:teuthology.orchestra.run.smithi031.stdout:3787 read 0x775e149 thru 0x7767658 (0x9510 bytes) 2024-09-13T07:13:10.471 INFO:teuthology.orchestra.run.smithi031.stdout:3788 read 0x701897d thru 0x702019c (0x7820 bytes) 2024-09-13T07:13:10.471 INFO:teuthology.orchestra.run.smithi031.stdout:3789 write 0x2eb55f0 thru 0x2ec2cb1 (0xd6c2 bytes) 2024-09-13T07:13:10.479 INFO:teuthology.orchestra.run.smithi031.stdout:3790 write 0x9cd540d thru 0x9cd9ea5 (0x4a99 bytes) 2024-09-13T07:13:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:3792 trunc from 0x9cd9ea6 to 0x91ac1d 2024-09-13T07:13:10.505 INFO:teuthology.orchestra.run.smithi031.stdout:3793 punch from 0x4fa1ef to 0x508907, (0xe718 bytes) 2024-09-13T07:13:10.516 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:10.516 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:10.516 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:10.516 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:10.518 INFO:teuthology.orchestra.run.smithi031.stdout:3794 writesame 0x6f0e6f4 thru 0x6f1d987 data_size 0x8a(0xf294 bytes) 2024-09-13T07:13:10.525 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:10.526 INFO:teuthology.orchestra.run.smithi031.stdout:3797 write 0x9445bbb thru 0x944c4c7 (0x690d bytes) 2024-09-13T07:13:10.528 INFO:teuthology.orchestra.run.smithi031.stdout:3798 punch from 0x2260bf1 to 0x226726b, (0x667a bytes) 2024-09-13T07:13:10.528 INFO:teuthology.orchestra.run.smithi031.stdout:3801 read 0xc3d88c thru 0xc4c4d8 (0xec4d bytes) 2024-09-13T07:13:10.529 INFO:teuthology.orchestra.run.smithi031.stdout:3803 trunc from 0x944c4c8 to 0x91cab66 2024-09-13T07:13:10.536 INFO:teuthology.orchestra.run.smithi031.stdout:3804 punch from 0x1715891 to 0x1715d56, (0x4c5 bytes) 2024-09-13T07:13:10.536 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.536 INFO:teuthology.orchestra.run.smithi031.stdout:3806 punch from 0xbeb83e to 0xbf441e, (0x8be0 bytes) 2024-09-13T07:13:10.536 INFO:teuthology.orchestra.run.smithi031.stdout:3808 read 0x3398d6a thru 0x33a6366 (0xd5fd bytes) 2024-09-13T07:13:10.536 INFO:teuthology.orchestra.run.smithi031.stdout:3810 read 0x51a6e14 thru 0x51b57de (0xe9cb bytes) 2024-09-13T07:13:10.536 INFO:teuthology.orchestra.run.smithi031.stdout:3811 write 0x3fbd9db thru 0x3fc3435 (0x5a5b bytes) 2024-09-13T07:13:10.542 INFO:teuthology.orchestra.run.smithi031.stdout:3812 writesame 0x39cdb9c thru 0x39d44b9 data_size 0x45(0x691e bytes) 2024-09-13T07:13:10.550 INFO:teuthology.orchestra.run.smithi031.stdout:3815 writesame 0x55681d9 thru 0x5571604 data_size 0x74(0x942c bytes) 2024-09-13T07:13:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:3817 punch from 0x6b630b7 to 0x6b709e4, (0xd92d bytes) 2024-09-13T07:13:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:3818 writesame 0x661b167 thru 0x6627dc5 data_size 0x1(0xcc5f bytes) 2024-09-13T07:13:10.568 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:10.568+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:10.577 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.577 INFO:teuthology.orchestra.run.smithi031.stdout:3821 read 0x2ed882 thru 0x2ef42f (0x1bae bytes) 2024-09-13T07:13:10.577 INFO:teuthology.orchestra.run.smithi031.stdout:3822 punch from 0x4e31ee0 to 0x4e3fbfc, (0xdd1c bytes) 2024-09-13T07:13:10.589 INFO:teuthology.orchestra.run.smithi031.stdout:3823 writesame 0xe546424 thru 0xe54c788 data_size 0x7(0x6365 bytes) 2024-09-13T07:13:10.589 INFO:teuthology.orchestra.run.smithi031.stdout:3824 write 0x3e96805 thru 0x3e9d23f (0x6a3b bytes) 2024-09-13T07:13:10.590 INFO:teuthology.orchestra.run.smithi031.stdout:3825 read 0xcad12df thru 0xcadf161 (0xde83 bytes) 2024-09-13T07:13:10.590 INFO:teuthology.orchestra.run.smithi031.stdout:3827 read 0xb3fe45 thru 0xb40b24 (0xce0 bytes) 2024-09-13T07:13:10.591 INFO:teuthology.orchestra.run.smithi031.stdout:3828 read 0x4665d4c thru 0x46727b1 (0xca66 bytes) 2024-09-13T07:13:10.591 INFO:teuthology.orchestra.run.smithi031.stdout:3829 punch from 0x563ac1a to 0x5648d50, (0xe136 bytes) 2024-09-13T07:13:10.591 INFO:teuthology.orchestra.run.smithi031.stdout:3832 write 0x6e10e9 thru 0x6e3d5a (0x2c72 bytes) 2024-09-13T07:13:10.593 INFO:teuthology.orchestra.run.smithi031.stdout:3833 punch from 0x4106b0e to 0x410802f, (0x1521 bytes) 2024-09-13T07:13:10.593 INFO:teuthology.orchestra.run.smithi031.stdout:3834 read 0xc83f6b3 thru 0xc84ba41 (0xc38f bytes) 2024-09-13T07:13:10.593 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.593 INFO:teuthology.orchestra.run.smithi031.stdout:3836 read 0x15b0aed thru 0x15b9439 (0x894d bytes) 2024-09-13T07:13:10.593 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:10.593 INFO:teuthology.orchestra.run.smithi031.stdout:3838 trunc from 0xe54c789 to 0xe7854b2 2024-09-13T07:13:10.598 INFO:teuthology.orchestra.run.smithi031.stdout:3839 clone 22 order 22 su 16384 sc 11 2024-09-13T07:13:10.668 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:10.668 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:10.668 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:10.668 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:10.668 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:10.669 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:10.669+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:10.677 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:10.770 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:10.771+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:10.782 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:10.782 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:10.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:10.782 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.744860361999997, 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:10.784 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.785 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.786 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:13:10.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.788 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1768 3.a e25/19 ondisk, result = 0)", 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.729987276999999, 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010131984, 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.789 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052064+0000", 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052072+0000", 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9529999999999999e-06 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.790 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052090+0000", 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8263000000000001e-05 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052102+0000", 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2078e-05 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062047+0000", 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0099446580000000003 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.791 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062059+0000", 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2455e-05 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062197+0000", 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013813200000000001 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21398 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:10.792 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.773325341, 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088116559999999993, 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21398 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:10.793 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.794 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008729+0000", 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.798e-06 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008734+0000", 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.763e-06 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008738+0000", 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7999999999999998e-06 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011757+0000", 2024-09-13T07:13:10.795 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030188580000000001 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011838+0000", 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1021999999999998e-05 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012183+0000", 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034440499999999999 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.796 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013964+0000", 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001781071 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017492+0000", 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035283329999999998 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017520+0000", 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7687000000000001e-05 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.797 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017539+0000", 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9126e-05 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21399 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.773310171, 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088642229999999992, 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.798 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21399 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.799 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008745+0000", 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008750+0000", 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008755+0000", 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6480000000000002e-06 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012580+0000", 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003825214 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012630+0000", 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9772999999999999e-05 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012831+0000", 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000201294 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013967+0000", 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001136176 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017566+0000", 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035988190000000001 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017590+0000", 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3490999999999999e-05 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017607+0000", 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21921 3.10 3:0f55d758:::rbd_data.1173f6773d36.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.803 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.74517588, 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065309330000000001, 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21921 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036876+0000", 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038933+0000", 2024-09-13T07:13:10.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020573900000000001 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038947+0000", 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4071e-05 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038955+0000", 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1839999999999999e-06 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038999+0000", 2024-09-13T07:13:10.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3399000000000002e-05 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039104+0000", 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104798 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039324+0000", 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00022014499999999999 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041941+0000", 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026169420000000001 2024-09-13T07:13:10.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043354+0000", 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001412792 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043385+0000", 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1767999999999998e-05 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043408+0000", 2024-09-13T07:13:10.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2422999999999999e-05 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22131 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.604094852999999, 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076826359999999996, 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22131 2024-09-13T07:13:10.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177957+0000", 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177960+0000", 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9079999999999999e-06 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177967+0000", 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4219999999999997e-06 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177970+0000", 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9380000000000001e-06 2024-09-13T07:13:10.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178788+0000", 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081823400000000004 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178835+0000", 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7348000000000001e-05 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179052+0000", 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021655000000000001 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.182995+0000", 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039430350000000001 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185592+0000", 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025971280000000002 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185618+0000", 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6390999999999999e-05 2024-09-13T07:13:10.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185640+0000", 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2144e-05 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22132 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.604066212999999, 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076953339999999999, 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.814 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22132 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177989+0000", 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6749999999999998e-06 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177995+0000", 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8839999999999999e-06 2024-09-13T07:13:10.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177999+0000", 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6650000000000002e-06 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179333+0000", 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001334029 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.817 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179381+0000", 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8115000000000003e-05 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179582+0000", 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020005199999999999 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183028+0000", 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003446185 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185646+0000", 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00261845 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185666+0000", 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0301000000000001e-05 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185682+0000", 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5333999999999999e-05 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22133 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.604049297, 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007715502, 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22133 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178005+0000", 2024-09-13T07:13:10.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.745e-06 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178008+0000", 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.354e-06 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178011+0000", 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.847e-06 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179904+0000", 2024-09-13T07:13:10.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001892828 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179977+0000", 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2959000000000004e-05 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180170+0000", 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019334699999999999 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183047+0000", 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028766690000000001 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185685+0000", 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002638315 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185704+0000", 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8929000000000001e-05 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185719+0000", 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4772e-05 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22134 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.60403913, 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077416129999999996, 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.826 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.826 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22134 2024-09-13T07:13:10.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.826 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:10.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178013+0000", 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178015+0000", 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.449e-06 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178018+0000", 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0970000000000002e-06 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178021+0000", 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9790000000000002e-06 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180446+0000", 2024-09-13T07:13:10.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002425207 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180494+0000", 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7870000000000001e-05 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180680+0000", 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018602899999999999 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183074+0000", 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023943660000000002 2024-09-13T07:13:10.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185722+0000", 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026480010000000001 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185741+0000", 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8328000000000001e-05 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185755+0000", 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4552999999999999e-05 2024-09-13T07:13:10.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22135 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.604029887999999, 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007772101, 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.832 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22135 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178024+0000", 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3909999999999999e-06 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178027+0000", 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178029+0000", 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4679999999999999e-06 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180968+0000", 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029389149999999998 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181015+0000", 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7178999999999997e-05 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181200+0000", 2024-09-13T07:13:10.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018424700000000001 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183091+0000", 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001891413 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185758+0000", 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026670819999999999 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185778+0000", 2024-09-13T07:13:10.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9656e-05 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185795+0000", 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7020999999999998e-05 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22384 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.387691344, 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078826639999999993, 2024-09-13T07:13:10.837 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22384 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:10.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394363+0000", 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.511e-06 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394368+0000", 2024-09-13T07:13:10.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4589999999999997e-06 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394371+0000", 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397357+0000", 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029860659999999999 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397406+0000", 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8982e-05 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397659+0000", 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025325 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.401969+0000", 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043103020000000002 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402177+0000", 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020735 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402215+0000", 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8532999999999999e-05 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402244+0000", 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8756999999999999e-05 2024-09-13T07:13:10.842 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22563 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.239991140000001, 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065088769999999997, 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22563 2024-09-13T07:13:10.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542061+0000", 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542063+0000", 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4810000000000001e-06 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542066+0000", 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.954e-06 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542068+0000", 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6280000000000001e-06 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546580+0000", 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045117259999999998 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546648+0000", 2024-09-13T07:13:10.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7398999999999996e-05 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546893+0000", 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024524099999999999 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548346+0000", 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001453153 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548486+0000", 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013988499999999999 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548532+0000", 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6675000000000003e-05 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548570+0000", 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7938000000000002e-05 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22607 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.206426186, 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067359300000000002, 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22607 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.849 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575629+0000", 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.943e-06 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575635+0000", 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0249999999999999e-06 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575640+0000", 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0900000000000004e-06 2024-09-13T07:13:10.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.578914+0000", 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032737140000000001 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579018+0000", 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000103928 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579262+0000", 2024-09-13T07:13:10.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000243988 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.581291+0000", 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020291419999999998 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582314+0000", 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010232660000000001 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582343+0000", 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8844e-05 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582362+0000", 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9409e-05 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22608 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:10.854 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.206408437, 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097116479999999998, 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22608 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575646+0000", 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:13:10.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575655+0000", 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3359999999999995e-06 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575660+0000", 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579853+0000", 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004193123 2024-09-13T07:13:10.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579939+0000", 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6316999999999994e-05 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580288+0000", 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034927800000000001 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582212+0000", 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019236679999999999 2024-09-13T07:13:10.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585304+0000", 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030922509999999999 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585337+0000", 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585356+0000", 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8899e-05 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.859 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22609 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.206381381, 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097598709999999998, 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22609 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.860 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575674+0000", 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0850000000000001e-06 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575685+0000", 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195e-05 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575690+0000", 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3589999999999999e-06 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580716+0000", 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050255289999999999 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580764+0000", 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8402000000000002e-05 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580983+0000", 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021850999999999999 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585136+0000", 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041530070000000002 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585385+0000", 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024898500000000002 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585412+0000", 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7240999999999999e-05 2024-09-13T07:13:10.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585431+0000", 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8995e-05 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22684 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.152084965, 2024-09-13T07:13:10.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069291209999999999, 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22684 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629967+0000", 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629969+0000", 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.468e-06 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629972+0000", 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0520000000000002e-06 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629975+0000", 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634438+0000", 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044633980000000004 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634515+0000", 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7020000000000005e-05 2024-09-13T07:13:10.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634768+0000", 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025338500000000001 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636038+0000", 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012701800000000001 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636808+0000", 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076938199999999999 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636854+0000", 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6149000000000001e-05 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636897+0000", 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2796999999999997e-05 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22685 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:10.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.152076081000001, 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072232559999999999, 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22685 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629976+0000", 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629978+0000", 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.42e-06 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629981+0000", 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7630000000000001e-06 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629983+0000", 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.424e-06 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635281+0000", 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052976969999999996 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635373+0000", 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2596000000000006e-05 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635650+0000", 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027690599999999999 2024-09-13T07:13:10.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636553+0000", 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000902646 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637134+0000", 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058095900000000003 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637169+0000", 2024-09-13T07:13:10.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4793999999999999e-05 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637200+0000", 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1254999999999998e-05 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.877 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22784 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.067034598999999, 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072328160000000004, 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22784 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:10.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715020+0000", 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715026+0000", 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9190000000000004e-06 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715031+0000", 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5889999999999996e-06 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720306+0000", 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005275475 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720370+0000", 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3802999999999999e-05 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720620+0000", 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025008499999999999 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721297+0000", 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00067708199999999997 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722163+0000", 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00086599500000000005 2024-09-13T07:13:10.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722217+0000", 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3393999999999997e-05 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722251+0000", 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4223000000000001e-05 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.883 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1979514899999999, 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:13:10.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:72295 3.a 3:521aa6cb:::rbd_data.1173ab534c79.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4325376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.666699+0000", 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1153541149999999, 2024-09-13T07:13:10.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014436838, 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 72295 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.666699+0000", 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.666699+0000", 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.666697+0000", 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674891+0000", 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0081942349999999994 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:10.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674915+0000", 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4247e-05 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674927+0000", 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.172e-05 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674976+0000", 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8668999999999998e-05 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.675093+0000", 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117106 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.675375+0000", 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00028205999999999999 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.679487+0000", 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041123109999999996 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.681025+0000", 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015378169999999999 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.681086+0000", 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1409999999999996e-05 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.681135+0000", 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9166999999999999e-05 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.894 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:10.895 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:10.895 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:10.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:10.897 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:10.898+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:10.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:10.998 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:10.999+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:11.008 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:11.008 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:11.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:11.023 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:11.041 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:11.075 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:11.092 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:11.099 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:11.100+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:11.110 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:11.110 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:11.126 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T07:13:11.130 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:13:11.130 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:13:11.131 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:11.132 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:11.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:11.200 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:11.201+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:11.288 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:11.288 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:11.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:11.340 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:11.341+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:13:11.402 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:11.403+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:11.411 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:11.411 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:11.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:11.428 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone21 2024-09-13T07:13:11.503 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:11.504+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:11.540 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:11.540 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:11.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:11.604 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:11.605+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:11.609 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T07:13:11.618 INFO:teuthology.orchestra.run.smithi031.stdout:3840 write 0x2e8e892 thru 0x2e93ea4 (0x5613 bytes) 2024-09-13T07:13:11.650 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:11.661 INFO:teuthology.orchestra.run.smithi031.stdout:3842 trunc from 0xe7854b2 to 0x45fd4a7 2024-09-13T07:13:11.676 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:11.676 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:11.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:11.705 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:11.706+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:11.806 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:11.807+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:11.815 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:11.815+0000 7efee3c12640 -1 osd.7 30 log_to_monitors true 2024-09-13T07:13:11.836 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:11.836 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:11.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:11.907 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:11.908+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:11.927 INFO:teuthology.orchestra.run.smithi031.stdout:3844 read 0x2e9d94d thru 0x2ea160d (0x3cc1 bytes) 2024-09-13T07:13:11.927 INFO:teuthology.orchestra.run.smithi031.stdout:3845 write 0x2e989f2 thru 0x2ea413f (0xb74e bytes) 2024-09-13T07:13:11.929 INFO:teuthology.orchestra.run.smithi031.stdout:3847 write 0x45193b thru 0x455ac3 (0x4189 bytes) 2024-09-13T07:13:11.976 INFO:teuthology.orchestra.run.smithi031.stdout:3848 write 0xd3919f5 thru 0xd393e14 (0x2420 bytes) 2024-09-13T07:13:11.979 INFO:teuthology.orchestra.run.smithi031.stdout:3849 writesame 0x6001ca6 thru 0x60079cb data_size 0x1(0x5d26 bytes) 2024-09-13T07:13:11.981 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:11.981 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:11.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:11.991 INFO:teuthology.orchestra.run.smithi031.stdout:3851 trunc from 0xd393e15 to 0xaab5725 2024-09-13T07:13:12.002 INFO:teuthology.orchestra.run.smithi031.stdout:3852 read 0x67a820e thru 0x67ad353 (0x5146 bytes) 2024-09-13T07:13:12.002 INFO:teuthology.orchestra.run.smithi031.stdout:3853 read 0x6486ec4 thru 0x64892e2 (0x241f bytes) 2024-09-13T07:13:12.002 INFO:teuthology.orchestra.run.smithi031.stdout:3854 write 0x58ef14 thru 0x59bc89 (0xcd76 bytes) 2024-09-13T07:13:12.008 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:12.009+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:12.058 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.067 INFO:teuthology.orchestra.run.smithi031.stdout:3858 write 0xddc281f thru 0xddcaba3 (0x8385 bytes) 2024-09-13T07:13:12.072 INFO:teuthology.orchestra.run.smithi031.stdout:3859 writesame 0x7735e4a thru 0x773b62e data_size 0x1(0x57e5 bytes) 2024-09-13T07:13:12.079 INFO:teuthology.orchestra.run.smithi031.stdout:3860 writesame 0x11be3c7 thru 0x11c71e5 data_size 0x1(0x8e1f bytes) 2024-09-13T07:13:12.088 INFO:teuthology.orchestra.run.smithi031.stdout:3861 write 0x69763c thru 0x6a215e (0xab23 bytes) 2024-09-13T07:13:12.090 INFO:teuthology.orchestra.run.smithi031.stdout:3862 write 0x2f67b6d thru 0x2f7745f (0xf8f3 bytes) 2024-09-13T07:13:12.092 INFO:teuthology.orchestra.run.smithi031.stdout:3863 read 0xacedc16 thru 0xacef1f7 (0x15e2 bytes) 2024-09-13T07:13:12.092 INFO:teuthology.orchestra.run.smithi031.stdout:3864 writesame 0xae1a282 thru 0xae204a3 data_size 0x6a(0x6222 bytes) 2024-09-13T07:13:12.106 INFO:teuthology.orchestra.run.smithi031.stdout:3867 write 0xd879be6 thru 0xd880971 (0x6d8c bytes) 2024-09-13T07:13:12.109 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:12.109+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:12.117 INFO:teuthology.orchestra.run.smithi031.stdout:3868 write 0xbeca9ba thru 0xbed3dc9 (0x9410 bytes) 2024-09-13T07:13:12.123 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:12.124 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:3869 writesame 0xc835a83 thru 0xc83a2fe data_size 0x2(0x487c bytes) 2024-09-13T07:13:12.133 INFO:teuthology.orchestra.run.smithi031.stdout:3870 write 0x816ab83 thru 0x816b6a3 (0xb21 bytes) 2024-09-13T07:13:12.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:12.137 INFO:teuthology.orchestra.run.smithi031.stdout:3871 write 0x19a98ca thru 0x19b8198 (0xe8cf bytes) 2024-09-13T07:13:12.156 INFO:teuthology.orchestra.run.smithi031.stdout:3872 read 0x40a5820 thru 0x40a6bfb (0x13dc bytes) 2024-09-13T07:13:12.157 INFO:teuthology.orchestra.run.smithi031.stdout:3873 read 0x95438c9 thru 0x95459e2 (0x211a bytes) 2024-09-13T07:13:12.157 INFO:teuthology.orchestra.run.smithi031.stdout:3876 writesame 0x1e90a05 thru 0x1e9e6f4 data_size 0x8c(0xdcf0 bytes) 2024-09-13T07:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout:3879 punch from 0x1fc04cc to 0x1fc6967, (0x649b bytes) 2024-09-13T07:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout:3884 clone 23 order 19 su 131072 sc 6 2024-09-13T07:13:12.210 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:12.210+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:12.259 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:12.259 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:12.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:12.310 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:12.311+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:12.354 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:12.355+0000 7efed7442640 -1 osd.7 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:13:12.400 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone22 from 0xddcaba4 (overlap 0x45fd4a7) to 0x14ff2fe 2024-09-13T07:13:12.410 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:12.411+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:12.434 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:12.434 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:12.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:12.493 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T07:13:12.511 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:12.512+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:12.612 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:12.613+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:12.615 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:12.615 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:12.625 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:12.713 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:12.714+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:12.728 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:12.728 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:12.736 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:12.760 INFO:teuthology.orchestra.run.smithi031.stdout:3885 write 0xedbe371 thru 0xedca92c (0xc5bc bytes) 2024-09-13T07:13:12.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.764 INFO:teuthology.orchestra.run.smithi031.stdout:3887 write 0xbed1201 thru 0xbed8a6f (0x786f bytes) 2024-09-13T07:13:12.770 INFO:teuthology.orchestra.run.smithi031.stdout:3889 read 0xa88572f thru 0xa8885e8 (0x2eba bytes) 2024-09-13T07:13:12.770 INFO:teuthology.orchestra.run.smithi031.stdout:3890 write 0xb5feb26 thru 0xb60a2c3 (0xb79e bytes) 2024-09-13T07:13:12.785 INFO:teuthology.orchestra.run.smithi031.stdout:3891 trunc from 0xedca92d to 0x5ebe815 2024-09-13T07:13:12.810 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:12.812 INFO:teuthology.orchestra.run.smithi031.stdout:3893 trunc from 0x5ebe815 to 0x16747b1 2024-09-13T07:13:12.813 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:12.814+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:12.840 INFO:teuthology.orchestra.run.smithi031.stdout:3895 punch from 0x7e6688 to 0x7e7cd5, (0x164d bytes) 2024-09-13T07:13:12.843 INFO:teuthology.orchestra.run.smithi031.stdout:3897 trunc from 0x16747b1 to 0x421c5a9 2024-09-13T07:13:12.854 INFO:teuthology.orchestra.run.smithi031.stdout:3898 write 0x44c4f32 thru 0x44d3107 (0xe1d6 bytes) 2024-09-13T07:13:12.857 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:12.857 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:12.858 INFO:teuthology.orchestra.run.smithi031.stdout:3903 write 0x349023 thru 0x3563ee (0xd3cc bytes) 2024-09-13T07:13:12.864 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.869 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:12.877 INFO:teuthology.orchestra.run.smithi031.stdout:3905 writesame 0xa8c1fb9 thru 0xa8c8dd1 data_size 0x5(0x6e19 bytes) 2024-09-13T07:13:12.877 INFO:teuthology.orchestra.run.smithi031.stdout:3907 write 0x513bd1e thru 0x51423d8 (0x66bb bytes) 2024-09-13T07:13:12.881 INFO:teuthology.orchestra.run.smithi031.stdout:3908 write 0x1ad746b thru 0x1ad9265 (0x1dfb bytes) 2024-09-13T07:13:12.884 INFO:teuthology.orchestra.run.smithi031.stdout:3909 read 0x4eeaf49 thru 0x4ef07c8 (0x5880 bytes) 2024-09-13T07:13:12.884 INFO:teuthology.orchestra.run.smithi031.stdout:3910 read 0x3709692 thru 0x3710133 (0x6aa2 bytes) 2024-09-13T07:13:12.884 INFO:teuthology.orchestra.run.smithi031.stdout:3913 punch from 0x9f4abd3 to 0x9f59107, (0xe534 bytes) 2024-09-13T07:13:12.884 INFO:teuthology.orchestra.run.smithi031.stdout:3915 read 0x4a18cc2 thru 0x4a26cba (0xdff9 bytes) 2024-09-13T07:13:12.892 INFO:teuthology.orchestra.run.smithi031.stdout:3916 write 0xc5a8884 thru 0xc5ae5a4 (0x5d21 bytes) 2024-09-13T07:13:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:3917 write 0xe623463 thru 0xe6261dc (0x2d7a bytes) 2024-09-13T07:13:12.914 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:12.915+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:12.918 INFO:teuthology.orchestra.run.smithi031.stdout:3918 trunc from 0xe6261dd to 0x3f4d5a9 2024-09-13T07:13:12.929 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:12.930 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:12.930 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:12.935 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.953 INFO:teuthology.orchestra.run.smithi031.stdout:3920 writesame 0xe9730a3 thru 0xe9750fc data_size 0x29(0x205a bytes) 2024-09-13T07:13:12.953 INFO:teuthology.orchestra.run.smithi031.stdout:3921 write 0xd908a80 thru 0xd909c4f (0x11d0 bytes) 2024-09-13T07:13:12.958 INFO:teuthology.orchestra.run.smithi031.stdout:3923 punch from 0x269a031 to 0x269d9eb, (0x39ba bytes) 2024-09-13T07:13:12.958 INFO:teuthology.orchestra.run.smithi031.stdout:3924 read 0xd303f52 thru 0xd3126a8 (0xe757 bytes) 2024-09-13T07:13:12.958 INFO:teuthology.orchestra.run.smithi031.stdout:3926 write 0xcaad49 thru 0xcb3d60 (0x9018 bytes) 2024-09-13T07:13:12.963 INFO:teuthology.orchestra.run.smithi031.stdout:3927 writesame 0x916173a thru 0x9163853 data_size 0x13(0x211a bytes) 2024-09-13T07:13:12.966 INFO:teuthology.orchestra.run.smithi031.stdout:3929 punch from 0x6108b1b to 0x61099c4, (0xea9 bytes) 2024-09-13T07:13:12.966 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.966 INFO:teuthology.orchestra.run.smithi031.stdout:3932 punch from 0xa6adb66 to 0xa6b715c, (0x95f6 bytes) 2024-09-13T07:13:12.966 INFO:teuthology.orchestra.run.smithi031.stdout:3933 write 0x9d74c11 thru 0x9d833d4 (0xe7c4 bytes) 2024-09-13T07:13:12.970 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.970 INFO:teuthology.orchestra.run.smithi031.stdout:3937 punch from 0xa327212 to 0xa32a3f4, (0x31e2 bytes) 2024-09-13T07:13:12.978 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:12.978 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:12.980 INFO:teuthology.orchestra.run.smithi031.stdout:3938 trunc from 0xe9750fd to 0x5073272 2024-09-13T07:13:12.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:12.990 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:12.995 INFO:teuthology.orchestra.run.smithi031.stdout:3940 trunc from 0x5073272 to 0xabb63a 2024-09-13T07:13:13.023 INFO:teuthology.orchestra.run.smithi031.stdout:3941 write 0x1878802 thru 0x1887d26 (0xf525 bytes) 2024-09-13T07:13:13.031 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:13.032+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:3942 trunc from 0x1887d27 to 0x9b6ebe8 2024-09-13T07:13:13.040 INFO:teuthology.orchestra.run.smithi031.stdout:3943 writesame 0x6b2777b thru 0x6b32483 data_size 0x1(0xad09 bytes) 2024-09-13T07:13:13.058 INFO:teuthology.orchestra.run.smithi031.stdout:3944 writesame 0x9b57674 thru 0x9b580f5 data_size 0x5(0xa82 bytes) 2024-09-13T07:13:13.061 INFO:teuthology.orchestra.run.smithi031.stdout:3946 punch from 0x967f591 to 0x968d491, (0xdf00 bytes) 2024-09-13T07:13:13.061 INFO:teuthology.orchestra.run.smithi031.stdout:3947 clone 24 order 20 su 8192 sc 8 2024-09-13T07:13:13.117 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:13.117+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:13.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:13.135 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:13.135 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:13.143 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:13.217 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:13.217+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:13.244 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:13.244 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:13.253 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:13.254 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:13.255 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:13.271 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:13.271 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:13.287 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:13.303 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:13.318 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:13.319+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:13.319 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:13.319 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:13.336 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:13.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:13:13.359 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:13.362 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:13.362 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:13.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:13.362 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:13.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.362 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36784 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13295616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698827517, 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028020702000000001, 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36784 2024-09-13T07:13:13.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:13.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663479+0000", 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1299999999999999e-06 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663485+0000", 2024-09-13T07:13:13.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0430000000000004e-06 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663494+0000", 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6220000000000004e-06 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691423+0000", 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027928339 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691464+0000", 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1743999999999999e-05 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691498+0000", 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3235999999999998e-05 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36785 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13320192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:13.367 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.69880594, 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028091378, 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36785 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663498+0000", 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663500+0000", 2024-09-13T07:13:13.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2400000000000002e-06 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663506+0000", 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663511+0000", 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6689999999999997e-06 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691518+0000", 2024-09-13T07:13:13.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028007641999999999 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691557+0000", 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8139000000000002e-05 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691590+0000", 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3334000000000003e-05 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36786 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13344768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698789551000001, 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028164842999999998, 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36786 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663514+0000", 2024-09-13T07:13:13.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663521+0000", 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663527+0000", 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8690000000000001e-06 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663532+0000", 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4229999999999998e-06 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691609+0000", 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028077429000000001 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691647+0000", 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7540000000000003e-05 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691680+0000", 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3161000000000003e-05 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36787 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13369344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698760597, 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028232916, 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.376 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36787 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663546+0000", 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3070000000000001e-06 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663552+0000", 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1149999999999999e-06 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663556+0000", 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691702+0000", 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028145752999999999 2024-09-13T07:13:13.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691741+0000", 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8611e-05 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691777+0000", 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5982999999999998e-05 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.380 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36788 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13393920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698744057999999, 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028311981999999999, 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36788 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663565+0000", 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4519999999999996e-06 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663571+0000", 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5480000000000001e-06 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663576+0000", 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6079999999999998e-06 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691800+0000", 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028224468999999999 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691839+0000", 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8476999999999998e-05 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691872+0000", 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3868000000000002e-05 2024-09-13T07:13:13.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36789 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13418496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698724925, 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028396060000000001, 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36789 2024-09-13T07:13:13.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663579+0000", 2024-09-13T07:13:13.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663581+0000", 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9369999999999998e-06 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663586+0000", 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2800000000000003e-06 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663590+0000", 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.031e-06 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691896+0000", 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028305815000000002 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691941+0000", 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4956999999999997e-05 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691976+0000", 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4422999999999999e-05 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36790 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13443072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698710026000001, 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028477820000000001, 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36790 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663596+0000", 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2390000000000001e-06 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663602+0000", 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6740000000000002e-06 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663607+0000", 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5159999999999996e-06 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691998+0000", 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028391937999999999 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692036+0000", 2024-09-13T07:13:13.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7948999999999999e-05 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692072+0000", 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5893999999999999e-05 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36791 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13467648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698694113, 2024-09-13T07:13:13.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028558888000000001, 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36791 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663612+0000", 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9420000000000002e-06 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663617+0000", 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.49e-06 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663630+0000", 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2126e-05 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692094+0000", 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028464382 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692134+0000", 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0234999999999997e-05 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692169+0000", 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5098999999999997e-05 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36792 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13492224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698670777, 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028626243999999999, 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36792 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663633+0000", 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663636+0000", 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1909999999999999e-06 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663641+0000", 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7089999999999999e-06 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663646+0000", 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8339999999999998e-06 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692191+0000", 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028544491000000002 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692227+0000", 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6189999999999997e-05 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692260+0000", 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3198999999999998e-05 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36793 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13516800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698654394, 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028699295999999999, 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36793 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663656+0000", 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6220000000000003e-06 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663663+0000", 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1489999999999999e-06 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663667+0000", 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5589999999999999e-06 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692280+0000", 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028612779000000001 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692316+0000", 2024-09-13T07:13:13.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6492999999999998e-05 2024-09-13T07:13:13.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692349+0000", 2024-09-13T07:13:13.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3052000000000001e-05 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.69863249, 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:13.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:13.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698594839, 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.411 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:13.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:13.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.698575611999999, 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:13.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36799 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13664256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.697375717, 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027993130000000001, 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36799 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664928+0000", 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664930+0000", 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0109999999999999e-06 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664936+0000", 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7810000000000002e-06 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664941+0000", 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5990000000000004e-06 2024-09-13T07:13:13.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692841+0000", 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027900115999999999 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692887+0000", 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6063999999999998e-05 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692922+0000", 2024-09-13T07:13:13.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4906000000000001e-05 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36800 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13688832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.697359757999999, 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028070522000000001, 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.425 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36800 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664944+0000", 2024-09-13T07:13:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664950+0000", 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4949999999999999e-06 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664956+0000", 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6829999999999996e-06 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664967+0000", 2024-09-13T07:13:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1858000000000001e-05 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692944+0000", 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027976671000000002 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692981+0000", 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7311999999999998e-05 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693015+0000", 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3856999999999998e-05 2024-09-13T07:13:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36801 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13713408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.697332715, 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028136613000000001, 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36801 2024-09-13T07:13:13.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664971+0000", 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664974+0000", 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4710000000000001e-06 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664980+0000", 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6620000000000002e-06 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664984+0000", 2024-09-13T07:13:13.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.561e-06 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693038+0000", 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028054374999999999 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693075+0000", 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6740999999999999e-05 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693108+0000", 2024-09-13T07:13:13.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3207000000000002e-05 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36802 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13737984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.697301827, 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028199977000000001, 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36802 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665002+0000", 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665005+0000", 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5790000000000002e-06 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665011+0000", 2024-09-13T07:13:13.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7089999999999996e-06 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665022+0000", 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0631e-05 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693129+0000", 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028106822 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693168+0000", 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9069999999999997e-05 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693203+0000", 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4826999999999997e-05 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.437 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36803 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13762560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.697278047999999, 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028271376000000001, 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36803 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665028+0000", 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.294e-06 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665034+0000", 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5559999999999998e-06 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665038+0000", 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5179999999999998e-06 2024-09-13T07:13:13.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693224+0000", 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028185387999999999 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693265+0000", 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0791999999999998e-05 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693298+0000", 2024-09-13T07:13:13.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3198000000000003e-05 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36804 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13787136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 11.697261774999999, 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028345479, 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36804 2024-09-13T07:13:13.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665042+0000", 2024-09-13T07:13:13.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665044+0000", 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.199e-06 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665050+0000", 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.733e-06 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665054+0000", 2024-09-13T07:13:13.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2119999999999997e-06 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693319+0000", 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028264105000000001 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693355+0000", 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6618000000000001e-05 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693388+0000", 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3065999999999998e-05 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.8225022000000006, 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:13.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:13.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.451 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:13.452 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:13.454 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-13T07:13:13.455 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:13.453+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:13.554 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:13.555+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:13.598 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:13.598 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:13.598 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:13.598 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:13.607 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:13.654 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:13.655+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:13.751 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:13.751 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:13.752 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:13.752 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:13.752 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:13.756 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:13.757+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:13.760 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:13.861 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:13.861+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:13.905 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:13.905 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:13.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:13.908 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:13.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.908 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:13:13.908 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:13.908 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.867489396000003, 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop_reply(client.4467.0:1768 3.a e25/19 ondisk, result = 0)", 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 35.852616310999998, 2024-09-13T07:13:13.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.010131984, 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052065+0000", 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052064+0000", 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052072+0000", 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9529999999999999e-06 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052090+0000", 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8263000000000001e-05 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.052102+0000", 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2078e-05 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062047+0000", 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0099446580000000003 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062059+0000", 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2455e-05 2024-09-13T07:13:13.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062197+0000", 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013813200000000001 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21398 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.895954374999999, 2024-09-13T07:13:13.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088116559999999993, 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21398 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008727+0000", 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008729+0000", 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.798e-06 2024-09-13T07:13:13.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008734+0000", 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.763e-06 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008738+0000", 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7999999999999998e-06 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011757+0000", 2024-09-13T07:13:13.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030188580000000001 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.011838+0000", 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1021999999999998e-05 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012183+0000", 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034440499999999999 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013964+0000", 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001781071 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017492+0000", 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035283329999999998 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017520+0000", 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7687000000000001e-05 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017539+0000", 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9126e-05 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21399 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.895939205000001, 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0088642229999999992, 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.924 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21399 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008742+0000", 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008745+0000", 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008750+0000", 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.008755+0000", 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6480000000000002e-06 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012580+0000", 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003825214 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012630+0000", 2024-09-13T07:13:13.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9772999999999999e-05 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.012831+0000", 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000201294 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.013967+0000", 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001136176 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017566+0000", 2024-09-13T07:13:13.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035988190000000001 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017590+0000", 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3490999999999999e-05 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.017607+0000", 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:21921 3.10 3:0f55d758:::rbd_data.1173f6773d36.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.867804914000001, 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065309330000000001, 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 21921 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.930 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036877+0000", 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.036876+0000", 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038933+0000", 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020573900000000001 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038947+0000", 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4071e-05 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038955+0000", 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1839999999999999e-06 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.038999+0000", 2024-09-13T07:13:13.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3399000000000002e-05 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039104+0000", 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000104798 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.039324+0000", 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00022014499999999999 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.041941+0000", 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026169420000000001 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043354+0000", 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001412792 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043385+0000", 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1767999999999998e-05 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.043408+0000", 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2422999999999999e-05 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22131 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.726723886999999, 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076826359999999996, 2024-09-13T07:13:13.936 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22131 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177958+0000", 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177957+0000", 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177960+0000", 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9079999999999999e-06 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177967+0000", 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4219999999999997e-06 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177970+0000", 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9380000000000001e-06 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178788+0000", 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081823400000000004 2024-09-13T07:13:13.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178835+0000", 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7348000000000001e-05 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179052+0000", 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021655000000000001 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.182995+0000", 2024-09-13T07:13:13.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039430350000000001 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185592+0000", 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025971280000000002 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185618+0000", 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6390999999999999e-05 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185640+0000", 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2144e-05 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22132 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.726695246999999, 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0076953339999999999, 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22132 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:13.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177986+0000", 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177989+0000", 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6749999999999998e-06 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177995+0000", 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8839999999999999e-06 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.177999+0000", 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6650000000000002e-06 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179333+0000", 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001334029 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179381+0000", 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8115000000000003e-05 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179582+0000", 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020005199999999999 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183028+0000", 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003446185 2024-09-13T07:13:13.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185646+0000", 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00261845 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185666+0000", 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0301000000000001e-05 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185682+0000", 2024-09-13T07:13:13.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5333999999999999e-05 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22133 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.726678330999999, 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007715502, 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.948 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22133 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178003+0000", 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178005+0000", 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.745e-06 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178008+0000", 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.354e-06 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178011+0000", 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.847e-06 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179904+0000", 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001892828 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.179977+0000", 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2959000000000004e-05 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180170+0000", 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00019334699999999999 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183047+0000", 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028766690000000001 2024-09-13T07:13:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185685+0000", 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002638315 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185704+0000", 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8929000000000001e-05 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185719+0000", 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4772e-05 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22134 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.726668163999999, 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0077416129999999996, 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.954 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22134 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178014+0000", 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178013+0000", 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178015+0000", 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.449e-06 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178018+0000", 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0970000000000002e-06 2024-09-13T07:13:13.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178021+0000", 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9790000000000002e-06 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180446+0000", 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002425207 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180494+0000", 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7870000000000001e-05 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180680+0000", 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018602899999999999 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183074+0000", 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0023943660000000002 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185722+0000", 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026480010000000001 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185741+0000", 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8328000000000001e-05 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185755+0000", 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4552999999999999e-05 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22135 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.726658921999999, 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.007772101, 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22135 2024-09-13T07:13:13.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178023+0000", 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178024+0000", 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3909999999999999e-06 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178027+0000", 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.178029+0000", 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4679999999999999e-06 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.180968+0000", 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029389149999999998 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181015+0000", 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7178999999999997e-05 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.181200+0000", 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018424700000000001 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.183091+0000", 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001891413 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185758+0000", 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026670819999999999 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185778+0000", 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9656e-05 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.185795+0000", 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7020999999999998e-05 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.965 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22384 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.510320377999999, 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0078826639999999993, 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22384 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394361+0000", 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394363+0000", 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.511e-06 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394368+0000", 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4589999999999997e-06 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.394371+0000", 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397357+0000", 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029860659999999999 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397406+0000", 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8982e-05 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.397659+0000", 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025325 2024-09-13T07:13:13.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.401969+0000", 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043103020000000002 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402177+0000", 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00020735 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402215+0000", 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8532999999999999e-05 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.402244+0000", 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8756999999999999e-05 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22563 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.362620174, 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0065088769999999997, 2024-09-13T07:13:13.971 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22563 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542062+0000", 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542061+0000", 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542063+0000", 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4810000000000001e-06 2024-09-13T07:13:13.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542066+0000", 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.954e-06 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.542068+0000", 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6280000000000001e-06 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546580+0000", 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045117259999999998 2024-09-13T07:13:13.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546648+0000", 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7398999999999996e-05 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.546893+0000", 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024524099999999999 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548346+0000", 2024-09-13T07:13:13.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001453153 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548486+0000", 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00013988499999999999 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548532+0000", 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6675000000000003e-05 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.548570+0000", 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7938000000000002e-05 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22607 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.329055220000001, 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0067359300000000002, 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.977 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22607 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:13.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575626+0000", 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575629+0000", 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.943e-06 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575635+0000", 2024-09-13T07:13:13.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0249999999999999e-06 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575640+0000", 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0900000000000004e-06 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.578914+0000", 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032737140000000001 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579018+0000", 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000103928 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579262+0000", 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000243988 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.581291+0000", 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020291419999999998 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582314+0000", 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010232660000000001 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582343+0000", 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8844e-05 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582362+0000", 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9409e-05 2024-09-13T07:13:13.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22608 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.329037470999999, 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097116479999999998, 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.983 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22608 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575644+0000", 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575646+0000", 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575655+0000", 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3359999999999995e-06 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575660+0000", 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579853+0000", 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004193123 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.579939+0000", 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6316999999999994e-05 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580288+0000", 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00034927800000000001 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.582212+0000", 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019236679999999999 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585304+0000", 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030922509999999999 2024-09-13T07:13:13.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585337+0000", 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585356+0000", 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8899e-05 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22609 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.329010414999999, 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0097598709999999998, 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22609 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575671+0000", 2024-09-13T07:13:13.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575674+0000", 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0850000000000001e-06 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575685+0000", 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1195e-05 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.575690+0000", 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3589999999999999e-06 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580716+0000", 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050255289999999999 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580764+0000", 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8402000000000002e-05 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.580983+0000", 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00021850999999999999 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585136+0000", 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041530070000000002 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585385+0000", 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00024898500000000002 2024-09-13T07:13:13.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585412+0000", 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7240999999999999e-05 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.585431+0000", 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8995e-05 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:13.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22684 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.274713998999999, 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0069291209999999999, 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:13.995 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22684 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629968+0000", 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629967+0000", 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629969+0000", 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.468e-06 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629972+0000", 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0520000000000002e-06 2024-09-13T07:13:13.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629975+0000", 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634438+0000", 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044633980000000004 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634515+0000", 2024-09-13T07:13:13.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7020000000000005e-05 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.634768+0000", 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025338500000000001 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636038+0000", 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012701800000000001 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636808+0000", 2024-09-13T07:13:13.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076938199999999999 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636854+0000", 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6149000000000001e-05 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636897+0000", 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2796999999999997e-05 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:14.000 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22685 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.274705115, 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072232559999999999, 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22685 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.001 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629977+0000", 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629976+0000", 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629978+0000", 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.42e-06 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629981+0000", 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7630000000000001e-06 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.629983+0000", 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.424e-06 2024-09-13T07:13:14.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635281+0000", 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052976969999999996 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635373+0000", 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.2596000000000006e-05 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.635650+0000", 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027690599999999999 2024-09-13T07:13:14.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.636553+0000", 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000902646 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637134+0000", 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00058095900000000003 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:14.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637169+0000", 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4793999999999999e-05 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.637200+0000", 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.1254999999999998e-05 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:22784 3.a 3:54a6577b:::rbd_object_map.1173f6773d36:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.189663632999999, 2024-09-13T07:13:14.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0072328160000000004, 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 22784 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:14.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715018+0000", 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715020+0000", 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:14.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715026+0000", 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9190000000000004e-06 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.715031+0000", 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5889999999999996e-06 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720306+0000", 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005275475 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720370+0000", 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3802999999999999e-05 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.720620+0000", 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00025008499999999999 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.721297+0000", 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00067708199999999997 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722163+0000", 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00086599500000000005 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722217+0000", 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3393999999999997e-05 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:55.722251+0000", 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4223000000000001e-05 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:14.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3205805240000004, 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.012 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:14.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:13:14.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:13:14.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:14.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:72295 3.a 3:521aa6cb:::rbd_data.1173ab534c79.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4325376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:14.017 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.666699+0000", 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.2379831489999997, 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014436838, 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 72295 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.666699+0000", 2024-09-13T07:13:14.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.666699+0000", 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.666697+0000", 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674891+0000", 2024-09-13T07:13:14.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0081942349999999994 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674915+0000", 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4247e-05 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674927+0000", 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.172e-05 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:14.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.674976+0000", 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8668999999999998e-05 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.675093+0000", 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000117106 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.675375+0000", 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00028205999999999999 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.679487+0000", 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041123109999999996 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.681025+0000", 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015378169999999999 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.681086+0000", 2024-09-13T07:13:14.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1409999999999996e-05 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.681135+0000", 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9166999999999999e-05 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:14.023 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:14.025 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:14.025 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:14.025+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:14.126 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:14.127+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:14.166 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:14.166 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.81394120000000003, 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.81396227099999996, 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "waiting for peered", 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:13:14.169 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-13T07:13:14.170 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 1 2024-09-13T07:13:14.171 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:14.175 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:14.226 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:14.227+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:14.276 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:14.276 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:14.276 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:14.276 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:14.276 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:14.285 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:14.328 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:14.329+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:14.348 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:13:14.400 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:14.400 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:14.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:14.401 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [] 2024-09-13T07:13:14.401 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:14.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:14.452 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:14.453+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:14.526 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:14.526 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:14.529 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:14.530+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:14.535 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:14.632 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:14.633+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:14.734 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:14.735 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:14.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:14.735 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:14.735+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:14.835 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:14.836+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:14.842 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51074 3.1f 3:fdb48563:::rbd_data.1173fee7d7bf.0000000000000006:15 [read 3387392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307972947, 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081542572999999993, 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51074 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535154+0000", 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0829999999999998e-06 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535161+0000", 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7139999999999996e-06 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535166+0000", 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.609803+0000", 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074637708999999997 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.609879+0000", 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5488999999999995e-05 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616695+0000", 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006816094 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51075 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307955384, 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081563839999999999, 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51075 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535172+0000", 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535178+0000", 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535183+0000", 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000001e-06 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616726+0000", 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081543051000000005 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616734+0000", 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51076 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307938384, 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081579246999999994, 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51076 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535189+0000", 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9709999999999998e-06 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535195+0000", 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0739999999999998e-06 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535199+0000", 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5549999999999996e-06 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616759+0000", 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081559926000000005 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616766+0000", 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51077 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307921766, 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081595972000000003, 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51077 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535203+0000", 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535210+0000", 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535217+0000", 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3490000000000003e-06 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535222+0000", 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616791+0000", 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081568733000000004 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616799+0000", 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.405e-06 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51078 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307898088, 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081603360999999999, 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51078 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535229+0000", 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535236+0000", 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.489e-06 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535240+0000", 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616823+0000", 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081582882999999995 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616831+0000", 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2280000000000001e-06 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51079 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307880635, 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081617985000000004, 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51079 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535244+0000", 2024-09-13T07:13:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535247+0000", 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535253+0000", 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2249999999999997e-06 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535258+0000", 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8899999999999998e-06 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616855+0000", 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081597307999999993 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616863+0000", 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6580000000000002e-06 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51080 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307862704, 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081642305999999998, 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51080 2024-09-13T07:13:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535262+0000", 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535264+0000", 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535275+0000", 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0936e-05 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535281+0000", 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616889+0000", 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081608705000000004 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616905+0000", 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5398999999999999e-05 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51081 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307839717, 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081649656000000001, 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51081 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535285+0000", 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535288+0000", 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535294+0000", 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535299+0000", 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616928+0000", 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081629060000000003 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616935+0000", 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.968e-06 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51082 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307821706, 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081660766999999995, 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51082 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535303+0000", 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535305+0000", 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535312+0000", 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535317+0000", 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616957+0000", 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081640502000000004 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616964+0000", 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-13T07:13:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51083 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307804072, 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081673832000000002, 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51083 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535323+0000", 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535329+0000", 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2650000000000002e-06 2024-09-13T07:13:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535334+0000", 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616988+0000", 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081653901000000001 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616995+0000", 2024-09-13T07:13:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1980000000000004e-06 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51084 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307786749, 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081686317999999994, 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51084 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535338+0000", 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535340+0000", 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0640000000000001e-06 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535347+0000", 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-13T07:13:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535351+0000", 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617017+0000", 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081665777999999994 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617025+0000", 2024-09-13T07:13:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5789999999999998e-06 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51085 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307769367000001, 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081702650000000002, 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51085 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535358+0000", 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535364+0000", 2024-09-13T07:13:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535369+0000", 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617051+0000", 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081681576000000006 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617059+0000", 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51086 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.307751573000001, 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081714234999999996, 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51086 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535373+0000", 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535376+0000", 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535382+0000", 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3899999999999998e-06 2024-09-13T07:13:14.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535387+0000", 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7559999999999999e-06 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617081+0000", 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081694514999999995 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617088+0000", 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:14.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51096 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.304872282, 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079140866000000004, 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51096 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538254+0000", 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538257+0000", 2024-09-13T07:13:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.847e-06 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538259+0000", 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2330000000000001e-06 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617387+0000", 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079127765000000003 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617394+0000", 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1659999999999997e-06 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51097 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.304864178000001, 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079164133999999997, 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51097 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538262+0000", 2024-09-13T07:13:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.203e-06 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538265+0000", 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538267+0000", 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617418+0000", 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079150707000000001 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617425+0000", 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2019999999999998e-06 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51098 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.304855848000001, 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079184118999999997, 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51098 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538270+0000", 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538273+0000", 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9040000000000001e-06 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538276+0000", 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5959999999999998e-06 2024-09-13T07:13:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617447+0000", 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079170849000000001 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617454+0000", 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8839999999999997e-06 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51100 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.304847351999999, 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079204752000000003, 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51100 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538279+0000", 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.097e-06 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538282+0000", 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538284+0000", 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4430000000000002e-06 2024-09-13T07:13:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617476+0000", 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079191704000000002 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617483+0000", 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9059999999999996e-06 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51102 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.304839027, 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079226071999999995, 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51102 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538287+0000", 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.099e-06 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538290+0000", 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.915e-06 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538292+0000", 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:13:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617505+0000", 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079212838999999993 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617512+0000", 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0299999999999996e-06 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51104 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.304830462, 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079248292999999997, 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51104 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538298+0000", 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0299999999999998e-06 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538301+0000", 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538303+0000", 2024-09-13T07:13:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617536+0000", 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079232494000000001 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617543+0000", 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4669999999999998e-06 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51106 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.304819963, 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079268452000000003, 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51106 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538306+0000", 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538309+0000", 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538312+0000", 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.458e-06 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617566+0000", 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079254597999999996 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617574+0000", 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:14.925 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:14.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:14.935 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:14.936+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:15.034 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:15.035 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:15.036 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:15.037+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:15.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:15.137 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:15.137+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:15.149 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:15.149 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:15.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:15.238 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:15.239+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49523 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2699264~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:15.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.863832106, 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056285390999999997, 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49523 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442063+0000", 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9379999999999999e-06 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442068+0000", 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442073+0000", 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497684+0000", 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055610930000000003 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497759+0000", 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5199999999999998e-05 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.498346+0000", 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058719100000000004 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49609 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 4075520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.845767775000001, 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055911012000000003, 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49609 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460127+0000", 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1430000000000001e-06 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460133+0000", 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5729999999999998e-06 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460138+0000", 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515783+0000", 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055645873999999998 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515861+0000", 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7979000000000007e-05 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.516036+0000", 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017504900000000001 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49618 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 999424~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.843253366999999, 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055816179, 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49618 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462639+0000", 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462642+0000", 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462647+0000", 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462652+0000", 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518291+0000", 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055639051000000002 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518368+0000", 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6913e-05 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518456+0000", 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8219000000000004e-05 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49619 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.843236942000001, 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055989537999999998, 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49619 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462659+0000", 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:13:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462665+0000", 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462669+0000", 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518483+0000", 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055813437 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518559+0000", 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5981000000000003e-05 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518646+0000", 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7175999999999993e-05 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49620 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1048576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.843219037000001, 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056152684000000001, 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49620 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462676+0000", 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:13:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462683+0000", 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462688+0000", 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518672+0000", 2024-09-13T07:13:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055983988999999998 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518747+0000", 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4633000000000002e-05 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518827+0000", 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000002e-05 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49621 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1073152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.843200289, 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056323023999999999, 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49621 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462695+0000", 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462701+0000", 2024-09-13T07:13:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462706+0000", 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6530000000000002e-06 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518854+0000", 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056147858000000002 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518934+0000", 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0813e-05 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519016+0000", 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1434999999999995e-05 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49622 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1097728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.843183003, 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056490088000000001, 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49622 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462712+0000", 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462718+0000", 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462722+0000", 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519042+0000", 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056320041000000001 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519120+0000", 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7607000000000004e-05 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519200+0000", 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166999999999993e-05 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49640 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1318912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.8393783, 2024-09-13T07:13:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056480083, 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49640 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466514+0000", 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466517+0000", 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466524+0000", 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466529+0000", 2024-09-13T07:13:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520731+0000", 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054202469000000003 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520808+0000", 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6854000000000007e-05 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522995+0000", 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00218681 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49653 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1490944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.839257088, 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055956026999999998, 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49653 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466638+0000", 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466643+0000", 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466648+0000", 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522424+0000", 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055775681000000001 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522499+0000", 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5828000000000007e-05 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522592+0000", 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2781000000000005e-05 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49674 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1662976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.837843699, 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056005177000000003, 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49674 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468052+0000", 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.497e-06 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468058+0000", 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2430000000000002e-06 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468063+0000", 2024-09-13T07:13:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523872+0000", 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055808706999999999 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523960+0000", 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8919999999999996e-05 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.524055+0000", 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4222999999999996e-05 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49695 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.835558038, 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056046466000000003, 2024-09-13T07:13:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49695 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470338+0000", 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470345+0000", 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4100000000000002e-06 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470351+0000", 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525756+0000", 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055404954999999999 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525819+0000", 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2779000000000003e-05 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526382+0000", 2024-09-13T07:13:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056291800000000003 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49696 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.83553734, 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056222355000000002, 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49696 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470355+0000", 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470358+0000", 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470364+0000", 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470369+0000", 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526411+0000", 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056041476999999999 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526489+0000", 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7787000000000006e-05 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526578+0000", 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9606999999999998e-05 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49704 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2056192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.833911489, 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055974318000000002, 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49704 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471981+0000", 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471984+0000", 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471990+0000", 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7229999999999998e-06 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471995+0000", 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527788+0000", 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055793091000000003 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527861+0000", 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3038000000000001e-05 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527956+0000", 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4536000000000004e-05 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49706 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2080768~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.833893681999999, 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056135461999999997, 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49706 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472006+0000", 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472012+0000", 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472017+0000", 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527980+0000", 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055963185999999998 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528050+0000", 2024-09-13T07:13:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979000000000002e-05 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528135+0000", 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4594999999999999e-05 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49708 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2105344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.833871521000001, 2024-09-13T07:13:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056284837999999997, 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49708 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472021+0000", 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472024+0000", 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.261e-06 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472029+0000", 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472034+0000", 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0069999999999997e-06 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528161+0000", 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056126285999999997 2024-09-13T07:13:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528232+0000", 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1000000000000005e-05 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528306+0000", 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4876000000000006e-05 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49711 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2129920~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.833854006999999, 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444877999999997, 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49711 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472041+0000", 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472047+0000", 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472052+0000", 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528331+0000", 2024-09-13T07:13:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056278504 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528399+0000", 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7941000000000006e-05 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528484+0000", 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5249999999999999e-05 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49713 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2154496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.833682694, 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444079000000001, 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49713 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472214+0000", 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472228+0000", 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3964999999999999e-05 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472236+0000", 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4109999999999999e-06 2024-09-13T07:13:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528509+0000", 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056273532000000001 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528581+0000", 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1338e-05 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528655+0000", 2024-09-13T07:13:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3858999999999999e-05 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49728 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2326528~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.831808793, 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055971704999999997, 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49728 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474087+0000", 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474093+0000", 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474098+0000", 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:13:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529880+0000", 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055782102 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529967+0000", 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6853000000000003e-05 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.530056+0000", 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9430000000000003e-05 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49767 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2695168~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.827719117000001, 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062028049000000002, 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49767 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478176+0000", 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478182+0000", 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478187+0000", 2024-09-13T07:13:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532704+0000", 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054517200000000002 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532779+0000", 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5244e-05 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.540202+0000", 2024-09-13T07:13:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074227429999999999 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:50078 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.813916937, 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055871918999999999, 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50078 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491979+0000", 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491986+0000", 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-13T07:13:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491991+0000", 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544160+0000", 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052168953999999997 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544241+0000", 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1213999999999999e-05 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.547848+0000", 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003606663 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:15.396 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:15.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:15.398 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:15.398+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:15.432 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone23 intact 2024-09-13T07:13:15.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:15.497 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:15.498+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:15.551 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:15.551 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:15.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:15.564 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T07:13:15.598 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:15.599+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:15.603 INFO:teuthology.orchestra.run.smithi031.stdout:3948 trunc from 0x9b6ebe8 to 0xdb6e834 2024-09-13T07:13:15.612 INFO:teuthology.orchestra.run.smithi031.stdout:3949 read 0x55e83ab thru 0x55ec71e (0x4374 bytes) 2024-09-13T07:13:15.612 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:15.612 INFO:teuthology.orchestra.run.smithi031.stdout:3951 write 0x1b200f3 thru 0x1b2b9df (0xb8ed bytes) 2024-09-13T07:13:15.622 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:15.622 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:15.628 INFO:teuthology.orchestra.run.smithi031.stdout:3952 punch from 0x5d8e68c to 0x5d943ff, (0x5d73 bytes) 2024-09-13T07:13:15.630 INFO:teuthology.orchestra.run.smithi031.stdout:3954 trunc from 0xdb6e834 to 0xec2caed 2024-09-13T07:13:15.635 INFO:teuthology.orchestra.run.smithi031.stdout:3955 write 0x22d0b0f thru 0x22dffa3 (0xf495 bytes) 2024-09-13T07:13:15.642 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:15.642 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:15.648 INFO:teuthology.orchestra.run.smithi031.stdout:3956 write 0xd770c6d thru 0xd779968 (0x8cfc bytes) 2024-09-13T07:13:15.659 INFO:teuthology.orchestra.run.smithi031.stdout:3957 read 0xd6b4d69 thru 0xd6b88d0 (0x3b68 bytes) 2024-09-13T07:13:15.659 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:15.660 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:15.674 INFO:teuthology.orchestra.run.smithi031.stdout:3959 trunc from 0xec2caed to 0x24e331b 2024-09-13T07:13:15.677 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:15.677 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:15.696 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:15.696 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:15.699 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:15.700+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:15.701 INFO:teuthology.orchestra.run.smithi031.stdout:3960 writesame 0x2eb201e thru 0x2eb36c9 data_size 0x2(0x16ac bytes) 2024-09-13T07:13:15.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:15.713 INFO:teuthology.orchestra.run.smithi031.stdout:3961 write 0x9785df1 thru 0x9787939 (0x1b49 bytes) 2024-09-13T07:13:15.716 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:15.716 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:15.717 INFO:teuthology.orchestra.run.smithi031.stdout:3962 read 0x593326b thru 0x593d9cc (0xa762 bytes) 2024-09-13T07:13:15.717 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:15.717 INFO:teuthology.orchestra.run.smithi031.stdout:3965 read 0x2915bf1 thru 0x291b865 (0x5c75 bytes) 2024-09-13T07:13:15.717 INFO:teuthology.orchestra.run.smithi031.stdout:3966 write 0xb4ce63 thru 0xb59586 (0xc724 bytes) 2024-09-13T07:13:15.745 INFO:teuthology.orchestra.run.smithi031.stdout:3967 write 0x681659b thru 0x6819dae (0x3814 bytes) 2024-09-13T07:13:15.745 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:15.745 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:15.747 INFO:teuthology.orchestra.run.smithi031.stdout:3968 punch from 0x3a9aaa3 to 0x3aa5f18, (0xb475 bytes) 2024-09-13T07:13:15.747 INFO:teuthology.orchestra.run.smithi031.stdout:3969 write 0x66f7901 thru 0x6704e31 (0xd531 bytes) 2024-09-13T07:13:15.761 INFO:teuthology.orchestra.run.smithi031.stdout:3970 read 0x345eaf9 thru 0x3468b9f (0xa0a7 bytes) 2024-09-13T07:13:15.762 INFO:teuthology.orchestra.run.smithi031.stdout:3971 punch from 0x2bb5e4f to 0x2bc3f43, (0xe0f4 bytes) 2024-09-13T07:13:15.762 INFO:teuthology.orchestra.run.smithi031.stdout:3972 punch from 0x86540e3 to 0x865dac1, (0x99de bytes) 2024-09-13T07:13:15.762 INFO:teuthology.orchestra.run.smithi031.stdout:3973 read 0x45ba095 thru 0x45c8c61 (0xebcd bytes) 2024-09-13T07:13:15.762 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:15.762 INFO:teuthology.orchestra.run.smithi031.stdout:3977 read 0x4813315 thru 0x481a648 (0x7334 bytes) 2024-09-13T07:13:15.762 INFO:teuthology.orchestra.run.smithi031.stdout:3978 write 0x3253a33 thru 0x3256203 (0x27d1 bytes) 2024-09-13T07:13:15.763 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:15.763 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:15.777 INFO:teuthology.orchestra.run.smithi031.stdout:3979 write 0x997f82b thru 0x9987c0b (0x83e1 bytes) 2024-09-13T07:13:15.787 INFO:teuthology.orchestra.run.smithi031.stdout:3980 write 0x3661f37 thru 0x366a835 (0x88ff bytes) 2024-09-13T07:13:15.793 INFO:teuthology.orchestra.run.smithi031.stdout:3981 read 0x8cde84 thru 0x8d5d7e (0x7efb bytes) 2024-09-13T07:13:15.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:15.794 INFO:teuthology.orchestra.run.smithi031.stdout:3987 read 0x92d0a34 thru 0x92d64bf (0x5a8c bytes) 2024-09-13T07:13:15.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:15.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:15.794 INFO:teuthology.orchestra.run.smithi031.stdout:3991 punch from 0x2bc4482 to 0x2bd06bf, (0xc23d bytes) 2024-09-13T07:13:15.800 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:15.801+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:15.803 INFO:teuthology.orchestra.run.smithi031.stdout:3993 writesame 0xd5cb7b7 thru 0xd5cc27e data_size 0x14(0xac8 bytes) 2024-09-13T07:13:15.803 INFO:teuthology.orchestra.run.smithi031.stdout:3995 write 0x1f110fd thru 0x1f20a7f (0xf983 bytes) 2024-09-13T07:13:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:3996 write 0xc3148ce thru 0xc3158c5 (0xff8 bytes) 2024-09-13T07:13:15.820 INFO:teuthology.orchestra.run.smithi031.stdout:3997 writesame 0xd6ad47a thru 0xd6b3577 data_size 0x41(0x60fe bytes) 2024-09-13T07:13:15.820 INFO:teuthology.orchestra.run.smithi031.stdout:3998 write 0x62ec24b thru 0x62f08fc (0x46b2 bytes) 2024-09-13T07:13:15.822 INFO:teuthology.orchestra.run.smithi031.stdout:3999 write 0x9adbae8 thru 0x9ae43da (0x88f3 bytes) 2024-09-13T07:13:15.831 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:15.831 INFO:teuthology.orchestra.run.smithi031.stdout:4001 write 0xefac4e thru 0xf0636a (0xb71d bytes) 2024-09-13T07:13:15.837 INFO:teuthology.orchestra.run.smithi031.stdout:4002 writesame 0x80fd739 thru 0x80fe936 data_size 0xe(0x11fe bytes) 2024-09-13T07:13:15.841 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:15.841 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:15.841 INFO:teuthology.orchestra.run.smithi031.stdout:4003 write 0x5bdd70f thru 0x5be995d (0xc24f bytes) 2024-09-13T07:13:15.849 INFO:teuthology.orchestra.run.smithi031.stdout:4004 write 0x180c240 thru 0x180d3ec (0x11ad bytes) 2024-09-13T07:13:15.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:15.850 INFO:teuthology.orchestra.run.smithi031.stdout:4006 writesame 0xc6cd5c4 thru 0xc6d8ecf data_size 0x1a(0xb90c bytes) 2024-09-13T07:13:15.858 INFO:teuthology.orchestra.run.smithi031.stdout:4008 clone 25 order 24 su 262144 sc 9 2024-09-13T07:13:15.901 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:15.902+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:15.964 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:15.964 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:15.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:16.002 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:16.003+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:16.103 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:16.104+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:16.118 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:16.118 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:16.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:16.204 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:16.205+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:16.272 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:16.272 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:16.282 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:16.326 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:16.327+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:16.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:13:16.390 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:16.390 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:16.398 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:16.412 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:16.413+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:16.507 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:16.507+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:16.541 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:16.541 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:16.549 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:16.608 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:16.609+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:16.652 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:16.652 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:16.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:16.744 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:16.745+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:16.803 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:16.804 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:16.810 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:16.811+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:16.812 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:16.911 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:16.912+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:16.954 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:16.954 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:16.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:17.047 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:17.048+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:17.105 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:17.105 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:17.114 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:17.114 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:17.114+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:17.214 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:17.215+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:17.257 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:17.257 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:17.266 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:17.315 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:17.316+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:13:17.413 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:17.414 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:17.415 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:17.417+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:17.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:17.442 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone24 from 0xd6b3578 (overlap 0x24e331b) to 0x0 2024-09-13T07:13:17.517 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:17.518+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:17.580 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T07:13:17.585 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:17.585 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:17.594 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:17.618 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:17.619+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:17.706 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:17.706 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:17.715 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:17.718 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:17.719+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:17.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:17.780 INFO:teuthology.orchestra.run.smithi031.stdout:4009 write 0xd981943 thru 0xd98caa5 (0xb163 bytes) 2024-09-13T07:13:17.792 INFO:teuthology.orchestra.run.smithi031.stdout:4011 writesame 0x691bc7f thru 0x6923c66 data_size 0x4(0x7fe8 bytes) 2024-09-13T07:13:17.819 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:17.820+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:17.826 INFO:teuthology.orchestra.run.smithi031.stdout:4012 writesame 0x212b20a thru 0x212b3ed data_size 0xb(0x1e4 bytes) 2024-09-13T07:13:17.828 INFO:teuthology.orchestra.run.smithi031.stdout:4013 punch from 0x456d26f to 0x4577852, (0xa5e3 bytes) 2024-09-13T07:13:17.828 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:17.828 INFO:teuthology.orchestra.run.smithi031.stdout:4017 write 0x4633d6e thru 0x46367f4 (0x2a87 bytes) 2024-09-13T07:13:17.834 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:17.834 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:17.844 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:17.859 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:17.861 INFO:teuthology.orchestra.run.smithi031.stdout:4019 trunc from 0xd98caa6 to 0xc19021d 2024-09-13T07:13:17.896 INFO:teuthology.orchestra.run.smithi031.stdout:4020 writesame 0x28a4a44 thru 0x28af418 data_size 0x1(0xa9d5 bytes) 2024-09-13T07:13:17.920 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:17.921+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:17.927 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:17.940 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:17.940 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:17.940 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-13T07:13:17.940 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:13:17.940 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:13:17.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-13T07:13:17.942 INFO:teuthology.orchestra.run.smithi031.stdout:4021 read 0x6d98e5b thru 0x6d9db8b (0x4d31 bytes) 2024-09-13T07:13:17.942 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:17.942 INFO:teuthology.orchestra.run.smithi031.stdout:4023 read 0x276e43a thru 0x276e8e0 (0x4a7 bytes) 2024-09-13T07:13:17.945 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:17.946 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:17.951 INFO:teuthology.orchestra.run.smithi031.stdout:4024 writesame 0xebcd0fd thru 0xebd0ae8 data_size 0x16(0x39ec bytes) 2024-09-13T07:13:17.952 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:17.952 INFO:teuthology.orchestra.run.smithi031.stdout:4028 write 0x5820f0f thru 0x58272f1 (0x63e3 bytes) 2024-09-13T07:13:17.954 INFO:teuthology.orchestra.run.smithi031.stdout:4029 write 0xbff8f3 thru 0xc0bef8 (0xc606 bytes) 2024-09-13T07:13:17.967 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:17.967 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:17.973 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:17.973 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:17.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:17.991 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:17.991 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:18.001 INFO:teuthology.orchestra.run.smithi031.stdout:4030 writesame 0x55a87e8 thru 0x55afe11 data_size 0x5(0x762a bytes) 2024-09-13T07:13:18.005 INFO:teuthology.orchestra.run.smithi031.stdout:4031 write 0x4eeddc4 thru 0x4ef4383 (0x65c0 bytes) 2024-09-13T07:13:18.007 INFO:teuthology.orchestra.run.smithi031.stdout:4034 writesame 0xbae8e3b thru 0xbaebbd5 data_size 0x5(0x2d9b bytes) 2024-09-13T07:13:18.009 INFO:teuthology.orchestra.run.smithi031.stdout:4035 write 0x8362447 thru 0x836e942 (0xc4fc bytes) 2024-09-13T07:13:18.011 INFO:teuthology.orchestra.run.smithi031.stdout:4038 read 0x68e3c17 thru 0x68e6bfa (0x2fe4 bytes) 2024-09-13T07:13:18.011 INFO:teuthology.orchestra.run.smithi031.stdout:4039 writesame 0xa3fdf68 thru 0xa407a51 data_size 0x4f(0x9aea bytes) 2024-09-13T07:13:18.013 INFO:teuthology.orchestra.run.smithi031.stdout:4040 punch from 0xae2bb0c to 0xae2f681, (0x3b75 bytes) 2024-09-13T07:13:18.013 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:18.013 INFO:teuthology.orchestra.run.smithi031.stdout:4043 read 0xa979ace thru 0xa982ec7 (0x93fa bytes) 2024-09-13T07:13:18.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:18.014 INFO:teuthology.orchestra.run.smithi031.stdout:4045 punch from 0x2cec812 to 0x2cf8897, (0xc085 bytes) 2024-09-13T07:13:18.014 INFO:teuthology.orchestra.run.smithi031.stdout:4048 write 0x463c3b0 thru 0x4642c7f (0x68d0 bytes) 2024-09-13T07:13:18.014 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:18.015 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:18.015 INFO:teuthology.orchestra.run.smithi031.stdout:4049 write 0xaf95d82 thru 0xaf96c41 (0xec0 bytes) 2024-09-13T07:13:18.017 INFO:teuthology.orchestra.run.smithi031.stdout:4050 read 0xd72a5ee thru 0xd732602 (0x8015 bytes) 2024-09-13T07:13:18.017 INFO:teuthology.orchestra.run.smithi031.stdout:4051 writesame 0x61bd9f5 thru 0x61caf73 data_size 0x11(0xd57f bytes) 2024-09-13T07:13:18.021 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:18.022+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:18.037 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:18.049 INFO:teuthology.orchestra.run.smithi031.stdout:4052 write 0x7024011 thru 0x702ab33 (0x6b23 bytes) 2024-09-13T07:13:18.073 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:18.081 INFO:teuthology.orchestra.run.smithi031.stdout:4054 write 0x48e1c9 thru 0x499097 (0xaecf bytes) 2024-09-13T07:13:18.083 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:18.083 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:13:18.083 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:18.083 INFO:teuthology.orchestra.run.smithi031.stdout:4055 read 0xab8c982 thru 0xab92711 (0x5d90 bytes) 2024-09-13T07:13:18.083 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:18.083 INFO:teuthology.orchestra.run.smithi031.stdout:4057 writesame 0x9a5fa7e thru 0x9a61774 data_size 0x1(0x1cf7 bytes) 2024-09-13T07:13:18.087 INFO:teuthology.orchestra.run.smithi031.stdout:4058 writesame 0xea131fa thru 0xea1bb27 data_size 0x9(0x892e bytes) 2024-09-13T07:13:18.090 INFO:teuthology.orchestra.run.smithi031.stdout:4059 write 0xaec4543 thru 0xaec801b (0x3ad9 bytes) 2024-09-13T07:13:18.106 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:18.106 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:18.106 INFO:teuthology.orchestra.run.smithi031.stdout:4062 trunc from 0xebd0ae9 to 0x26753f1 2024-09-13T07:13:18.134 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:18.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:18.135 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:18.135 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:18.135 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:18.136+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:18.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:18.163 INFO:teuthology.orchestra.run.smithi031.stdout:4064 write 0x72bca1f thru 0x72c7250 (0xa832 bytes) 2024-09-13T07:13:18.165 INFO:teuthology.orchestra.run.smithi031.stdout:4065 read 0x571f33c thru 0x572eef1 (0xfbb6 bytes) 2024-09-13T07:13:18.166 INFO:teuthology.orchestra.run.smithi031.stdout:4066 read 0x3e2dd22 thru 0x3e347d7 (0x6ab6 bytes) 2024-09-13T07:13:18.166 INFO:teuthology.orchestra.run.smithi031.stdout:4068 punch from 0x4423549 to 0x442ba7f, (0x8536 bytes) 2024-09-13T07:13:18.166 INFO:teuthology.orchestra.run.smithi031.stdout:4069 write 0x5b264c4 thru 0x5b2d521 (0x705e bytes) 2024-09-13T07:13:18.167 INFO:teuthology.orchestra.run.smithi031.stdout:4070 write 0xfedc42 thru 0xffaa28 (0xcde7 bytes) 2024-09-13T07:13:18.180 INFO:teuthology.orchestra.run.smithi031.stdout:4071 write 0xb8bd1db thru 0xb8c53cf (0x81f5 bytes) 2024-09-13T07:13:18.186 INFO:teuthology.orchestra.run.smithi031.stdout:4072 write 0x3d44c87 thru 0x3d4ade8 (0x6162 bytes) 2024-09-13T07:13:18.188 INFO:teuthology.orchestra.run.smithi031.stdout:4073 write 0xb2aff95 thru 0xb2b45aa (0x4616 bytes) 2024-09-13T07:13:18.192 INFO:teuthology.orchestra.run.smithi031.stdout:4076 writesame 0x66d7eb5 thru 0x66d9ff8 data_size 0x2(0x2144 bytes) 2024-09-13T07:13:18.193 INFO:teuthology.orchestra.run.smithi031.stdout:4077 punch from 0x5ac5804 to 0x5ac6923, (0x111f bytes) 2024-09-13T07:13:18.193 INFO:teuthology.orchestra.run.smithi031.stdout:4078 writesame 0x9f00060 thru 0x9f079c7 data_size 0x78(0x7968 bytes) 2024-09-13T07:13:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:4079 read 0x9e73813 thru 0x9e8332c (0xfb1a bytes) 2024-09-13T07:13:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:4080 write 0xc6f99a thru 0xc7bcbe (0xc325 bytes) 2024-09-13T07:13:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:4081 punch from 0x2997c11 to 0x29a6f20, (0xf30f bytes) 2024-09-13T07:13:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:4083 write 0x1ed607d thru 0x1ed9451 (0x33d5 bytes) 2024-09-13T07:13:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:4085 punch from 0x95bfc6 to 0x963e93, (0x7ecd bytes) 2024-09-13T07:13:18.206 INFO:teuthology.orchestra.run.smithi031.stdout:4086 write 0xc406d93 thru 0xc40c3ee (0x565c bytes) 2024-09-13T07:13:18.207 INFO:teuthology.orchestra.run.smithi031.stdout:4087 write 0x3f22a84 thru 0x3f29b29 (0x70a6 bytes) 2024-09-13T07:13:18.209 INFO:teuthology.orchestra.run.smithi031.stdout:4088 writesame 0x37f712d thru 0x37ff39c data_size 0x8(0x8270 bytes) 2024-09-13T07:13:18.211 INFO:teuthology.orchestra.run.smithi031.stdout:4089 read 0xa8d96b9 thru 0xa8de113 (0x4a5b bytes) 2024-09-13T07:13:18.212 INFO:teuthology.orchestra.run.smithi031.stdout:4091 clone 26 order 20 su 262144 sc 6 2024-09-13T07:13:18.223 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:18.224+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:18.273 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:18.273 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:18.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:18.324 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:18.325+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:18.425 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:18.425+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51074 3.1f 3:fdb48563:::rbd_data.1173fee7d7bf.0000000000000006:15 [read 3387392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.900046358999999, 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081542572999999993, 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51074 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535152+0000", 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535154+0000", 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0829999999999998e-06 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535161+0000", 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7139999999999996e-06 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535166+0000", 2024-09-13T07:13:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.609803+0000", 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074637708999999997 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.609879+0000", 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5488999999999995e-05 2024-09-13T07:13:18.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616695+0000", 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006816094 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51075 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.900028796000001, 2024-09-13T07:13:18.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081563839999999999, 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51075 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535172+0000", 2024-09-13T07:13:18.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535178+0000", 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535183+0000", 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000001e-06 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616726+0000", 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081543051000000005 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616734+0000", 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51076 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.900011795999999, 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081579246999999994, 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51076 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535189+0000", 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9709999999999998e-06 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535195+0000", 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0739999999999998e-06 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535199+0000", 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5549999999999996e-06 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616759+0000", 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081559926000000005 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616766+0000", 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51077 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899995177999999, 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081595972000000003, 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51077 2024-09-13T07:13:18.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535203+0000", 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535210+0000", 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535217+0000", 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3490000000000003e-06 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535222+0000", 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616791+0000", 2024-09-13T07:13:18.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081568733000000004 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616799+0000", 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.405e-06 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51078 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899971499999999, 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081603360999999999, 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51078 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535229+0000", 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535236+0000", 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.489e-06 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535240+0000", 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:18.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616823+0000", 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081582882999999995 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616831+0000", 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2280000000000001e-06 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51079 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899954047, 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081617985000000004, 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51079 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535244+0000", 2024-09-13T07:13:18.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535247+0000", 2024-09-13T07:13:18.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:18.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535253+0000", 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2249999999999997e-06 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535258+0000", 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8899999999999998e-06 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616855+0000", 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081597307999999993 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616863+0000", 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6580000000000002e-06 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51080 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899936115999999, 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081642305999999998, 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51080 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535262+0000", 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535264+0000", 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535275+0000", 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0936e-05 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535281+0000", 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616889+0000", 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081608705000000004 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616905+0000", 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5398999999999999e-05 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51081 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899913129, 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081649656000000001, 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51081 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535285+0000", 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535288+0000", 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535294+0000", 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535299+0000", 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616928+0000", 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081629060000000003 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616935+0000", 2024-09-13T07:13:18.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.968e-06 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51082 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899895118, 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081660766999999995, 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51082 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535303+0000", 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535305+0000", 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535312+0000", 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535317+0000", 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616957+0000", 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081640502000000004 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616964+0000", 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51083 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899877483999999, 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081673832000000002, 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51083 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535323+0000", 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535329+0000", 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2650000000000002e-06 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535334+0000", 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616988+0000", 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081653901000000001 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616995+0000", 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1980000000000004e-06 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51084 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899860160999999, 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081686317999999994, 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51084 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535338+0000", 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535340+0000", 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0640000000000001e-06 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535347+0000", 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535351+0000", 2024-09-13T07:13:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617017+0000", 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081665777999999994 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617025+0000", 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5789999999999998e-06 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51085 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899842779, 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081702650000000002, 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51085 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535358+0000", 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535364+0000", 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535369+0000", 2024-09-13T07:13:18.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617051+0000", 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081681576000000006 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617059+0000", 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51086 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.899824985, 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081714234999999996, 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51086 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535373+0000", 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535376+0000", 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535382+0000", 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3899999999999998e-06 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535387+0000", 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7559999999999999e-06 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617081+0000", 2024-09-13T07:13:18.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081694514999999995 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617088+0000", 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51096 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.896945693999999, 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079140866000000004, 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51096 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538253+0000", 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538254+0000", 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538257+0000", 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.847e-06 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538259+0000", 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2330000000000001e-06 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617387+0000", 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079127765000000003 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617394+0000", 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1659999999999997e-06 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51097 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.89693759, 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079164133999999997, 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51097 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538261+0000", 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538262+0000", 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.203e-06 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538265+0000", 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:13:18.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538267+0000", 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617418+0000", 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079150707000000001 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617425+0000", 2024-09-13T07:13:18.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2019999999999998e-06 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51098 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.89692926, 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079184118999999997, 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51098 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:18.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538269+0000", 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538270+0000", 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538273+0000", 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9040000000000001e-06 2024-09-13T07:13:18.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538276+0000", 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5959999999999998e-06 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617447+0000", 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079170849000000001 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617454+0000", 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8839999999999997e-06 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51100 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.896920764000001, 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079204752000000003, 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51100 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:18.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538278+0000", 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538279+0000", 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.097e-06 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538282+0000", 2024-09-13T07:13:18.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538284+0000", 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4430000000000002e-06 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617476+0000", 2024-09-13T07:13:18.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079191704000000002 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617483+0000", 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9059999999999996e-06 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51102 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.896912438999999, 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079226071999999995, 2024-09-13T07:13:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51102 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538286+0000", 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538287+0000", 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.099e-06 2024-09-13T07:13:18.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538290+0000", 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.915e-06 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538292+0000", 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617505+0000", 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079212838999999993 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617512+0000", 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0299999999999996e-06 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51104 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.896903873999999, 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079248292999999997, 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51104 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538295+0000", 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538298+0000", 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0299999999999998e-06 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538301+0000", 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538303+0000", 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T07:13:18.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617536+0000", 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079232494000000001 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617543+0000", 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4669999999999998e-06 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51106 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.896893374999999, 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079268452000000003, 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51106 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538305+0000", 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538306+0000", 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538309+0000", 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.538312+0000", 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.458e-06 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617566+0000", 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079254597999999996 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617574+0000", 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.517 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:18.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:18.519 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone25 from 0xc40c3ef (overlap 0x26753f1) to 0x0 2024-09-13T07:13:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T07:13:18.528 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:18.529+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:18.541 INFO:teuthology.orchestra.run.smithi031.stdout:4092 trunc from 0xc40c3ef to 0x15d7445 2024-09-13T07:13:18.569 INFO:teuthology.orchestra.run.smithi031.stdout:4093 read 0x3b417d thru 0x3c1e44 (0xdcc8 bytes) 2024-09-13T07:13:18.570 INFO:teuthology.orchestra.run.smithi031.stdout:4094 write 0x10eb685 thru 0x10ecad2 (0x144e bytes) 2024-09-13T07:13:18.576 INFO:teuthology.orchestra.run.smithi031.stdout:4095 punch from 0x7fd3a6 to 0x804245, (0x6e9f bytes) 2024-09-13T07:13:18.595 INFO:teuthology.orchestra.run.smithi031.stdout:4097 writesame 0xa06a376 thru 0xa06ebad data_size 0x4(0x4838 bytes) 2024-09-13T07:13:18.596 INFO:teuthology.orchestra.run.smithi031.stdout:4098 trunc from 0xa06ebae to 0x93f1889 2024-09-13T07:13:18.602 INFO:teuthology.orchestra.run.smithi031.stdout:4099 read 0x7514129 thru 0x751526f (0x1147 bytes) 2024-09-13T07:13:18.603 INFO:teuthology.orchestra.run.smithi031.stdout:4100 flatten 2024-09-13T07:13:18.628 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:18.628+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:18.643 INFO:teuthology.orchestra.run.smithi031.stdout:4102 trunc from 0x93f1889 to 0x9f5db66 2024-09-13T07:13:18.653 INFO:teuthology.orchestra.run.smithi031.stdout:4103 trunc from 0x9f5db66 to 0x64e104a 2024-09-13T07:13:18.662 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:18.662 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:18.662 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:18.663 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:18.674 INFO:teuthology.orchestra.run.smithi031.stdout:4105 write 0xe249d27 thru 0xe24eaca (0x4da4 bytes) 2024-09-13T07:13:18.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:4106 read 0x4f4afb thru 0x4f51e8 (0x6ee bytes) 2024-09-13T07:13:18.685 INFO:teuthology.orchestra.run.smithi031.stdout:4107 trunc from 0xe24eacb to 0x77fe4ca 2024-09-13T07:13:18.702 INFO:teuthology.orchestra.run.smithi031.stdout:4108 trunc from 0x77fe4ca to 0xc2a93c8 2024-09-13T07:13:18.708 INFO:teuthology.orchestra.run.smithi031.stdout:4109 writesame 0x2809b8e thru 0x28178b1 data_size 0x2(0xdd24 bytes) 2024-09-13T07:13:18.728 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:18.729+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:18.733 INFO:teuthology.orchestra.run.smithi031.stdout:4110 writesame 0xd9a7bab thru 0xd9b190e data_size 0xe(0x9d64 bytes) 2024-09-13T07:13:18.733 INFO:teuthology.orchestra.run.smithi031.stdout:4113 read 0xca720b7 thru 0xca7d51e (0xb468 bytes) 2024-09-13T07:13:18.733 INFO:teuthology.orchestra.run.smithi031.stdout:4114 punch from 0x39ffcc6 to 0x3a018a3, (0x1bdd bytes) 2024-09-13T07:13:18.733 INFO:teuthology.orchestra.run.smithi031.stdout:4117 write 0xe9176d thru 0xe97e9d (0x6731 bytes) 2024-09-13T07:13:18.736 INFO:teuthology.orchestra.run.smithi031.stdout:4118 write 0x1308cba thru 0x13153b0 (0xc6f7 bytes) 2024-09-13T07:13:18.738 INFO:teuthology.orchestra.run.smithi031.stdout:4119 write 0x98b18f3 thru 0x98b8bb5 (0x72c3 bytes) 2024-09-13T07:13:18.744 INFO:teuthology.orchestra.run.smithi031.stdout:4120 read 0x99b43de thru 0x99c000b (0xbc2e bytes) 2024-09-13T07:13:18.749 INFO:teuthology.orchestra.run.smithi031.stdout:4121 trunc from 0xd9b190f to 0x936ecab 2024-09-13T07:13:18.759 INFO:teuthology.orchestra.run.smithi031.stdout:4122 read 0x1854604 thru 0x1860843 (0xc240 bytes) 2024-09-13T07:13:18.764 INFO:teuthology.orchestra.run.smithi031.stdout:4123 trunc from 0x936ecab to 0xd51b927 2024-09-13T07:13:18.769 INFO:teuthology.orchestra.run.smithi031.stdout:4124 read 0x7996aea thru 0x79a28cc (0xbde3 bytes) 2024-09-13T07:13:18.769 INFO:teuthology.orchestra.run.smithi031.stdout:4125 write 0xc1cba8a thru 0xc1d0734 (0x4cab bytes) 2024-09-13T07:13:18.772 INFO:teuthology.orchestra.run.smithi031.stdout:4127 write 0x9371179 thru 0x937aeca (0x9d52 bytes) 2024-09-13T07:13:18.783 INFO:teuthology.orchestra.run.smithi031.stdout:4128 trunc from 0xd51b927 to 0x60fd00a 2024-09-13T07:13:18.792 INFO:teuthology.orchestra.run.smithi031.stdout:4129 read 0x4b3402a thru 0x4b4178f (0xd766 bytes) 2024-09-13T07:13:18.798 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:18.798 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:18.798 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:18.798 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:18.798 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:18.805 INFO:teuthology.orchestra.run.smithi031.stdout:4130 write 0xbe87117 thru 0xbe8d3af (0x6299 bytes) 2024-09-13T07:13:18.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:18.809 INFO:teuthology.orchestra.run.smithi031.stdout:4131 writesame 0x3a0b3c1 thru 0x3a0bf53 data_size 0x1(0xb93 bytes) 2024-09-13T07:13:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:4133 write 0xbded29e thru 0xbdf02c9 (0x302c bytes) 2024-09-13T07:13:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:4134 punch from 0x27161ca to 0x271b4be, (0x52f4 bytes) 2024-09-13T07:13:18.815 INFO:teuthology.orchestra.run.smithi031.stdout:4135 read 0x5ed8f8d thru 0x5edef47 (0x5fbb bytes) 2024-09-13T07:13:18.817 INFO:teuthology.orchestra.run.smithi031.stdout:4136 trunc from 0xbe8d3b0 to 0x9e7f858 2024-09-13T07:13:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:4137 writesame 0x573effc thru 0x57410d2 data_size 0x1(0x20d7 bytes) 2024-09-13T07:13:18.828 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:18.829+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:18.841 INFO:teuthology.orchestra.run.smithi031.stdout:4138 trunc from 0x9e7f858 to 0x24d6517 2024-09-13T07:13:18.859 INFO:teuthology.orchestra.run.smithi031.stdout:4139 trunc from 0x24d6517 to 0xbbcfce 2024-09-13T07:13:18.877 INFO:teuthology.orchestra.run.smithi031.stdout:4140 write 0x1f7fbef thru 0x1f8957e (0x9990 bytes) 2024-09-13T07:13:18.881 INFO:teuthology.orchestra.run.smithi031.stdout:4144 read 0x1e9702b thru 0x1ea4f9d (0xdf73 bytes) 2024-09-13T07:13:18.897 INFO:teuthology.orchestra.run.smithi031.stdout:4145 write 0xba9edb9 thru 0xbaae1a0 (0xf3e8 bytes) 2024-09-13T07:13:18.901 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:18.901 INFO:teuthology.orchestra.run.smithi031.stdout:4148 trunc from 0xbaae1a1 to 0xc2158a2 2024-09-13T07:13:18.906 INFO:teuthology.orchestra.run.smithi031.stdout:4149 write 0x1a0bf35 thru 0x1a100ae (0x417a bytes) 2024-09-13T07:13:18.910 INFO:teuthology.orchestra.run.smithi031.stdout:4151 write 0x95c9c18 thru 0x95d807e (0xe467 bytes) 2024-09-13T07:13:18.913 INFO:teuthology.orchestra.run.smithi031.stdout:4152 writesame 0xa8da149 thru 0xa8df717 data_size 0x1(0x55cf bytes) 2024-09-13T07:13:18.924 INFO:teuthology.orchestra.run.smithi031.stdout:4153 write 0x507895e thru 0x5080491 (0x7b34 bytes) 2024-09-13T07:13:18.927 INFO:teuthology.orchestra.run.smithi031.stdout:4154 read 0x3eb43d5 thru 0x3ebf076 (0xaca2 bytes) 2024-09-13T07:13:18.929 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:18.930+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:18.933 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49523 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2699264~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.491259875000001, 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056285390999999997, 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49523 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442063+0000", 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9379999999999999e-06 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442068+0000", 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442073+0000", 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497684+0000", 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055610930000000003 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497759+0000", 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5199999999999998e-05 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.498346+0000", 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058719100000000004 2024-09-13T07:13:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49609 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 4075520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.473195543999999, 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055911012000000003, 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49609 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460127+0000", 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1430000000000001e-06 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460133+0000", 2024-09-13T07:13:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5729999999999998e-06 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460138+0000", 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515783+0000", 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055645873999999998 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515861+0000", 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7979000000000007e-05 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.516036+0000", 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017504900000000001 2024-09-13T07:13:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49618 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 999424~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.470681136, 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055816179, 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49618 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462639+0000", 2024-09-13T07:13:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462642+0000", 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462647+0000", 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462652+0000", 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518291+0000", 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055639051000000002 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518368+0000", 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6913e-05 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518456+0000", 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8219000000000004e-05 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49619 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.470664711, 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055989537999999998, 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49619 2024-09-13T07:13:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462659+0000", 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462665+0000", 2024-09-13T07:13:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462669+0000", 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518483+0000", 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055813437 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518559+0000", 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5981000000000003e-05 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518646+0000", 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7175999999999993e-05 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49620 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1048576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.470646806, 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056152684000000001, 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49620 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462676+0000", 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462683+0000", 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462688+0000", 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518672+0000", 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055983988999999998 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518747+0000", 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4633000000000002e-05 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518827+0000", 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000002e-05 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49621 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1073152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.470628058000001, 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056323023999999999, 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49621 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462695+0000", 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462701+0000", 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462706+0000", 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6530000000000002e-06 2024-09-13T07:13:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518854+0000", 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056147858000000002 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518934+0000", 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0813e-05 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519016+0000", 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1434999999999995e-05 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49622 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1097728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.470610772000001, 2024-09-13T07:13:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056490088000000001, 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49622 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462712+0000", 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462718+0000", 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462722+0000", 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519042+0000", 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056320041000000001 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519120+0000", 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7607000000000004e-05 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519200+0000", 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166999999999993e-05 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49640 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1318912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.466806069, 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056480083, 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49640 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466514+0000", 2024-09-13T07:13:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466517+0000", 2024-09-13T07:13:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466524+0000", 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466529+0000", 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520731+0000", 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054202469000000003 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520808+0000", 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6854000000000007e-05 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522995+0000", 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00218681 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49653 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1490944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.466684857000001, 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055956026999999998, 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49653 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466638+0000", 2024-09-13T07:13:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466643+0000", 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466648+0000", 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522424+0000", 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055775681000000001 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522499+0000", 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5828000000000007e-05 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522592+0000", 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2781000000000005e-05 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49674 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1662976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.465271467999999, 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056005177000000003, 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49674 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468052+0000", 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.497e-06 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468058+0000", 2024-09-13T07:13:18.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2430000000000002e-06 2024-09-13T07:13:18.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468063+0000", 2024-09-13T07:13:18.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T07:13:18.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523872+0000", 2024-09-13T07:13:18.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055808706999999999 2024-09-13T07:13:18.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523960+0000", 2024-09-13T07:13:18.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8919999999999996e-05 2024-09-13T07:13:18.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.524055+0000", 2024-09-13T07:13:18.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4222999999999996e-05 2024-09-13T07:13:18.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.983 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49695 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.462985807000001, 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056046466000000003, 2024-09-13T07:13:18.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49695 2024-09-13T07:13:18.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.986 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:18.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:18.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470338+0000", 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470345+0000", 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4100000000000002e-06 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470351+0000", 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525756+0000", 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055404954999999999 2024-09-13T07:13:18.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525819+0000", 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2779000000000003e-05 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526382+0000", 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056291800000000003 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49696 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.462965109000001, 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056222355000000002, 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49696 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470355+0000", 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470358+0000", 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470364+0000", 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470369+0000", 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:18.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526411+0000", 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056041476999999999 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526489+0000", 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7787000000000006e-05 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526578+0000", 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9606999999999998e-05 2024-09-13T07:13:18.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49704 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2056192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.461339258000001, 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055974318000000002, 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:18.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49704 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471981+0000", 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471984+0000", 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471990+0000", 2024-09-13T07:13:18.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7229999999999998e-06 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471995+0000", 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527788+0000", 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055793091000000003 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527861+0000", 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3038000000000001e-05 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527956+0000", 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4536000000000004e-05 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49706 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2080768~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.461321451, 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056135461999999997, 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49706 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472006+0000", 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472012+0000", 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472017+0000", 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527980+0000", 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055963185999999998 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528050+0000", 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979000000000002e-05 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528135+0000", 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4594999999999999e-05 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49708 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2105344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.461299289999999, 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056284837999999997, 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49708 2024-09-13T07:13:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472021+0000", 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472024+0000", 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.261e-06 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472029+0000", 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472034+0000", 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0069999999999997e-06 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528161+0000", 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056126285999999997 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528232+0000", 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1000000000000005e-05 2024-09-13T07:13:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528306+0000", 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4876000000000006e-05 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49711 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2129920~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.461281776, 2024-09-13T07:13:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444877999999997, 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49711 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472041+0000", 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-13T07:13:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472047+0000", 2024-09-13T07:13:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-09-13T07:13:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472052+0000", 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528331+0000", 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056278504 2024-09-13T07:13:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528399+0000", 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7941000000000006e-05 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528484+0000", 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5249999999999999e-05 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49713 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2154496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.461110463000001, 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444079000000001, 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49713 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472214+0000", 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472228+0000", 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3964999999999999e-05 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472236+0000", 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4109999999999999e-06 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528509+0000", 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056273532000000001 2024-09-13T07:13:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528581+0000", 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1338e-05 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528655+0000", 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3858999999999999e-05 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49728 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2326528~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.459236561999999, 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055971704999999997, 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49728 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474087+0000", 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474093+0000", 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474098+0000", 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529880+0000", 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055782102 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529967+0000", 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6853000000000003e-05 2024-09-13T07:13:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.530056+0000", 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9430000000000003e-05 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49767 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2695168~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.455146886, 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062028049000000002, 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49767 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478176+0000", 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478182+0000", 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478187+0000", 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532704+0000", 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054517200000000002 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532779+0000", 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5244e-05 2024-09-13T07:13:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.540202+0000", 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074227429999999999 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:50078 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.441344706000001, 2024-09-13T07:13:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055871918999999999, 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50078 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491979+0000", 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491986+0000", 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491991+0000", 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544160+0000", 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052168953999999997 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544241+0000", 2024-09-13T07:13:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1213999999999999e-05 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.547848+0000", 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003606663 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.033 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:19.034 INFO:teuthology.orchestra.run.smithi031.stdout:4155 write 0xc85935d thru 0xc866827 (0xd4cb bytes) 2024-09-13T07:13:19.034 INFO:teuthology.orchestra.run.smithi031.stdout:4156 read 0xb9eac97 thru 0xb9eb986 (0xcf0 bytes) 2024-09-13T07:13:19.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4157 write 0xd07c087 thru 0xd07c5f8 (0x572 bytes) 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4158 writesame 0x466bfe thru 0x46efa6 data_size 0x9(0x83a9 bytes) 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4159 trunc from 0xd07c5f9 to 0x550f5b9 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4161 read 0x314e6db thru 0x315a2f8 (0xbc1e bytes) 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4162 write 0xdd0cebf thru 0xdd17655 (0xa797 bytes) 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4164 writesame 0x54eebc0 thru 0x54fcec6 data_size 0x1(0xe307 bytes) 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4165 write 0x7966d9c thru 0x7970f3f (0xa1a4 bytes) 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4166 write 0x212da3f thru 0x21329f5 (0x4fb7 bytes) 2024-09-13T07:13:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:4167 write 0x24efd09 thru 0x24f563b (0x5933 bytes) 2024-09-13T07:13:19.037 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:19.035+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:19.038 INFO:teuthology.orchestra.run.smithi031.stdout:4168 write 0xe732e60 thru 0xe73c9ec (0x9b8d bytes) 2024-09-13T07:13:19.041 INFO:teuthology.orchestra.run.smithi031.stdout:4169 punch from 0x48e6fdf to 0x48f5aea, (0xeb0b bytes) 2024-09-13T07:13:19.041 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.041 INFO:teuthology.orchestra.run.smithi031.stdout:4173 trunc from 0xe73c9ed to 0xdfb3fbe 2024-09-13T07:13:19.056 INFO:teuthology.orchestra.run.smithi031.stdout:4175 read 0xcd7a5fa thru 0xcd7e08a (0x3a91 bytes) 2024-09-13T07:13:19.056 INFO:teuthology.orchestra.run.smithi031.stdout:4177 writesame 0x5990af5 thru 0x59997e1 data_size 0x1(0x8ced bytes) 2024-09-13T07:13:19.088 INFO:teuthology.orchestra.run.smithi031.stdout:4178 write 0xe698055 thru 0xe6a4266 (0xc212 bytes) 2024-09-13T07:13:19.094 INFO:teuthology.orchestra.run.smithi031.stdout:4180 writesame 0x4961d81 thru 0x4964b49 data_size 0x1(0x2dc9 bytes) 2024-09-13T07:13:19.102 INFO:teuthology.orchestra.run.smithi031.stdout:4182 punch from 0xcb01dd4 to 0xcb0a388, (0x85b4 bytes) 2024-09-13T07:13:19.102 INFO:teuthology.orchestra.run.smithi031.stdout:4183 write 0x41175df thru 0x4120719 (0x913b bytes) 2024-09-13T07:13:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:4184 write 0x3914bba thru 0x391fa25 (0xae6c bytes) 2024-09-13T07:13:19.127 INFO:teuthology.orchestra.run.smithi031.stdout:4186 trunc from 0xe6a4267 to 0x52bb09b 2024-09-13T07:13:19.135 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:19.136+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:19.151 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:19.152 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:19.154 INFO:teuthology.orchestra.run.smithi031.stdout:4187 trunc from 0x52bb09b to 0xb1258b2 2024-09-13T07:13:19.161 INFO:teuthology.orchestra.run.smithi031.stdout:4188 read 0x67a3d0b thru 0x67a7200 (0x34f6 bytes) 2024-09-13T07:13:19.162 INFO:teuthology.orchestra.run.smithi031.stdout:4190 read 0x51479ee thru 0x514f789 (0x7d9c bytes) 2024-09-13T07:13:19.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:19.170 INFO:teuthology.orchestra.run.smithi031.stdout:4193 write 0xcf7329e thru 0xcf7724c (0x3faf bytes) 2024-09-13T07:13:19.173 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.173 INFO:teuthology.orchestra.run.smithi031.stdout:4196 trunc from 0xcf7724d to 0xcdc07c8 2024-09-13T07:13:19.179 INFO:teuthology.orchestra.run.smithi031.stdout:4197 punch from 0x69df17f to 0x69e26d4, (0x3555 bytes) 2024-09-13T07:13:19.179 INFO:teuthology.orchestra.run.smithi031.stdout:4199 read 0x5650227 thru 0x565b41e (0xb1f8 bytes) 2024-09-13T07:13:19.179 INFO:teuthology.orchestra.run.smithi031.stdout:4200 read 0xbb48e7c thru 0xbb4c6dc (0x3861 bytes) 2024-09-13T07:13:19.179 INFO:teuthology.orchestra.run.smithi031.stdout:4201 write 0x237d06a thru 0x238c189 (0xf120 bytes) 2024-09-13T07:13:19.187 INFO:teuthology.orchestra.run.smithi031.stdout:4202 writesame 0x673a013 thru 0x673e50d data_size 0x1(0x44fb bytes) 2024-09-13T07:13:19.211 INFO:teuthology.orchestra.run.smithi031.stdout:4203 trunc from 0xcdc07c8 to 0x10c1ce1 2024-09-13T07:13:19.230 INFO:teuthology.orchestra.run.smithi031.stdout:4204 writesame 0xb850fa thru 0xb8c130 data_size 0x1(0x7037 bytes) 2024-09-13T07:13:19.236 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:19.237+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:19.261 INFO:teuthology.orchestra.run.smithi031.stdout:4205 write 0x9b0d5e0 thru 0x9b13722 (0x6143 bytes) 2024-09-13T07:13:19.275 INFO:teuthology.orchestra.run.smithi031.stdout:4207 writesame 0xe129c2a thru 0xe12dd8a data_size 0x7(0x4161 bytes) 2024-09-13T07:13:19.275 INFO:teuthology.orchestra.run.smithi031.stdout:4208 writesame 0xf2ddbc thru 0xf306fd data_size 0x1(0x2942 bytes) 2024-09-13T07:13:19.283 INFO:teuthology.orchestra.run.smithi031.stdout:4209 writesame 0xb2f9315 thru 0xb300d79 data_size 0x1(0x7a65 bytes) 2024-09-13T07:13:19.287 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:19.287 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:19.288 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:19.288 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:19.288 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:19.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:19.298 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.301 INFO:teuthology.orchestra.run.smithi031.stdout:4211 trunc from 0xe12dd8b to 0xb046534 2024-09-13T07:13:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:4213 read 0x9dc6e0a thru 0x9dd624f (0xf446 bytes) 2024-09-13T07:13:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:4215 read 0x1a18896 thru 0x1a24e29 (0xc594 bytes) 2024-09-13T07:13:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:4216 read 0x80ef9f2 thru 0x80f3002 (0x3611 bytes) 2024-09-13T07:13:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:4218 punch from 0x28ad61 to 0x299a5e, (0xecfd bytes) 2024-09-13T07:13:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:4222 write 0x79d5073 thru 0x79d78ae (0x283c bytes) 2024-09-13T07:13:19.312 INFO:teuthology.orchestra.run.smithi031.stdout:4223 read 0x7664754 thru 0x766e3d6 (0x9c83 bytes) 2024-09-13T07:13:19.312 INFO:teuthology.orchestra.run.smithi031.stdout:4225 read 0x14875d5 thru 0x1495789 (0xe1b5 bytes) 2024-09-13T07:13:19.322 INFO:teuthology.orchestra.run.smithi031.stdout:4227 write 0xe3fe38b thru 0xe4005e7 (0x225d bytes) 2024-09-13T07:13:19.326 INFO:teuthology.orchestra.run.smithi031.stdout:4228 write 0x1a4bb3c thru 0x1a510f5 (0x55ba bytes) 2024-09-13T07:13:19.328 INFO:teuthology.orchestra.run.smithi031.stdout:4229 write 0x292b839 thru 0x29388ad (0xd075 bytes) 2024-09-13T07:13:19.331 INFO:teuthology.orchestra.run.smithi031.stdout:4231 punch from 0x7297199 to 0x72a69a5, (0xf80c bytes) 2024-09-13T07:13:19.332 INFO:teuthology.orchestra.run.smithi031.stdout:4234 punch from 0xb731e1 to 0xb7da21, (0xa840 bytes) 2024-09-13T07:13:19.332 INFO:teuthology.orchestra.run.smithi031.stdout:4235 read 0x7e5ace0 thru 0x7e685c5 (0xd8e6 bytes) 2024-09-13T07:13:19.332 INFO:teuthology.orchestra.run.smithi031.stdout:4236 punch from 0xb6e84f2 to 0xb6ed5b5, (0x50c3 bytes) 2024-09-13T07:13:19.333 INFO:teuthology.orchestra.run.smithi031.stdout:4239 trunc from 0xe4005e8 to 0xcc7985e 2024-09-13T07:13:19.337 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:19.338+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:19.345 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.347 INFO:teuthology.orchestra.run.smithi031.stdout:4241 trunc from 0xcc7985e to 0xebe07cf 2024-09-13T07:13:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:4242 punch from 0xe24d2d1 to 0xe25aeea, (0xdc19 bytes) 2024-09-13T07:13:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:4243 punch from 0xa4ef9d7 to 0xa4f28cc, (0x2ef5 bytes) 2024-09-13T07:13:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:4244 write 0x3ea7b13 thru 0x3eb6d5f (0xf24d bytes) 2024-09-13T07:13:19.371 INFO:teuthology.orchestra.run.smithi031.stdout:4245 trunc from 0xebe07cf to 0x327f23f 2024-09-13T07:13:19.384 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:13:19.387 INFO:teuthology.orchestra.run.smithi031.stdout:4247 trunc from 0x327f23f to 0x169bad9 2024-09-13T07:13:19.407 INFO:teuthology.orchestra.run.smithi031.stdout:4249 writesame 0x418b693 thru 0x419a753 data_size 0xd(0xf0c1 bytes) 2024-09-13T07:13:19.407 INFO:teuthology.orchestra.run.smithi031.stdout:4250 punch from 0x727cd8 to 0x727f61, (0x289 bytes) 2024-09-13T07:13:19.420 INFO:teuthology.orchestra.run.smithi031.stdout:4251 writesame 0x9d7a88b thru 0x9d81a94 data_size 0xb(0x720a bytes) 2024-09-13T07:13:19.420 INFO:teuthology.orchestra.run.smithi031.stdout:4255 write 0x9bf7081 thru 0x9bff5cd (0x854d bytes) 2024-09-13T07:13:19.422 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.422 INFO:teuthology.orchestra.run.smithi031.stdout:4257 punch from 0x56eaf53 to 0x56f8263, (0xd310 bytes) 2024-09-13T07:13:19.423 INFO:teuthology.orchestra.run.smithi031.stdout:4258 write 0x34acd71 thru 0x34b8a55 (0xbce5 bytes) 2024-09-13T07:13:19.424 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:19.424 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:19.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:19.424 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.126310801999999, 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:19.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.124832777000002, 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:19.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.122015728000001, 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.552586767999998, 2024-09-13T07:13:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.539963172, 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:13:19.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:13:19.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.796381404000002, 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:13:19.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.795844176999999, 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.455 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:13:19.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.656762540999999, 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:19.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4162 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'113, mlcod=29'110)", 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.609954823000002, 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028573499999999998, 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814277+0000", 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814283+0000", 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814293+0000", 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0299000000000001e-05 2024-09-13T07:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814298+0000", 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814345+0000", 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7707999999999998e-05 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814365+0000", 2024-09-13T07:13:19.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9836000000000001e-05 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817100+0000", 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027346969999999999 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817135+0000", 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5194999999999998e-05 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.609415247999998, 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:19.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.581931142999998, 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:13:19.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:13:19.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.575916718999999, 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:19.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.524224425, 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:13:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.988000577000001, 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7730 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'61, mlcod=34'54)", 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.987098798000002, 2024-09-13T07:13:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028499979999999999, 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437134+0000", 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437133+0000", 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437139+0000", 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7409999999999998e-06 2024-09-13T07:13:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437151+0000", 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1946e-05 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437157+0000", 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437201+0000", 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3917e-05 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.437221+0000", 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439958+0000", 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027367300000000002 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439984+0000", 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5551999999999999e-05 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:13:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.227847399000002, 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:13:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:13:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.71743871, 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:64537 3.9 e47/19 3:97a74db9:::rbd_data.117332306d66.0000000000000002:head v 47'372, mlcod=45'370)", 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.804825277999999, 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092884539999999998, 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.624988+0000", 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055812279999999997 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625018+0000", 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9496999999999999e-05 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625031+0000", 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625078+0000", 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7290000000000003e-05 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625096+0000", 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8091999999999999e-05 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628662+0000", 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035652449999999999 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628696+0000", 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4035999999999997e-05 2024-09-13T07:13:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:71538 3.0 e50/19 3:014540de:::rbd_data.1173f6918137.0000000000000082:head v 50'331, mlcod=49'326)", 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.005677579, 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032256870000000001, 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418554+0000", 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418562+0000", 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4000000000000003e-06 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418573+0000", 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1685000000000001e-05 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418580+0000", 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9419999999999998e-06 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418611+0000", 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1053000000000002e-05 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418630+0000", 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421749+0000", 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031190620000000001 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421781+0000", 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1905e-05 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:85070 3.0 e58/19 3:077f600c:::rbd_data.11732319a2dc.0000000000000001:head v 58'356, mlcod=57'354)", 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.80841285200000002, 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029694840000000001, 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615818+0000", 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616402+0000", 2024-09-13T07:13:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058395300000000001 2024-09-13T07:13:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616417+0000", 2024-09-13T07:13:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4630000000000001e-05 2024-09-13T07:13:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616424+0000", 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616469+0000", 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4508000000000001e-05 2024-09-13T07:13:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616499+0000", 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0491999999999999e-05 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618737+0000", 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002238037 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618789+0000", 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2055999999999998e-05 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.527 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:19.528 INFO:teuthology.orchestra.run.smithi031.stdout:4259 trunc from 0x9d81a95 to 0x81be719 2024-09-13T07:13:19.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4260 trunc from 0x81be719 to 0x6f01b8a 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4261 read 0x1d5977c thru 0x1d65a35 (0xc2ba bytes) 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4262 trunc from 0x6f01b8a to 0xdbded29 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4264 trunc from 0xdbded29 to 0x7892e39 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4265 write 0x7f34a20 thru 0x7f394ff (0x4ae0 bytes) 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4266 write 0x6b6a70c thru 0x6b75a03 (0xb2f8 bytes) 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4267 read 0x9f1d98 thru 0x9f3aff (0x1d68 bytes) 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4268 trunc from 0x7f39500 to 0x7a02bab 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4269 read 0x5000441 thru 0x500e892 (0xe452 bytes) 2024-09-13T07:13:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:4270 read 0x344cee4 thru 0x3451a81 (0x4b9e bytes) 2024-09-13T07:13:19.532 INFO:teuthology.orchestra.run.smithi031.stdout:4271 trunc from 0x7a02bab to 0x8a70617 2024-09-13T07:13:19.532 INFO:teuthology.orchestra.run.smithi031.stdout:4272 write 0x7ac9a08 thru 0x7acdad5 (0x40ce bytes) 2024-09-13T07:13:19.532 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.532 INFO:teuthology.orchestra.run.smithi031.stdout:4277 write 0xbf1fa0e thru 0xbf1fc57 (0x24a bytes) 2024-09-13T07:13:19.532 INFO:teuthology.orchestra.run.smithi031.stdout:4278 trunc from 0xbf1fc58 to 0x360b5ed 2024-09-13T07:13:19.532 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:19.529+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:19.541 INFO:teuthology.orchestra.run.smithi031.stdout:4279 trunc from 0x360b5ed to 0x795e70f 2024-09-13T07:13:19.546 INFO:teuthology.orchestra.run.smithi031.stdout:4280 writesame 0x17ad321 thru 0x17b7c44 data_size 0x32(0xa924 bytes) 2024-09-13T07:13:19.550 INFO:teuthology.orchestra.run.smithi031.stdout:4284 punch from 0x68678cc to 0x686f622, (0x7d56 bytes) 2024-09-13T07:13:19.550 INFO:teuthology.orchestra.run.smithi031.stdout:4285 punch from 0x3c52f87 to 0x3c60c2f, (0xdca8 bytes) 2024-09-13T07:13:19.550 INFO:teuthology.orchestra.run.smithi031.stdout:4286 read 0xc8e5df thru 0xc99eb8 (0xb8da bytes) 2024-09-13T07:13:19.551 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.551 INFO:teuthology.orchestra.run.smithi031.stdout:4289 read 0xd59f43 thru 0xd5f219 (0x52d7 bytes) 2024-09-13T07:13:19.560 INFO:teuthology.orchestra.run.smithi031.stdout:4290 write 0xa33bd95 thru 0xa3405a3 (0x480f bytes) 2024-09-13T07:13:19.565 INFO:teuthology.orchestra.run.smithi031.stdout:4292 write 0x77dd8c7 thru 0x77e56d8 (0x7e12 bytes) 2024-09-13T07:13:19.569 INFO:teuthology.orchestra.run.smithi031.stdout:4294 read 0x46999ae thru 0x469b9a7 (0x1ffa bytes) 2024-09-13T07:13:19.569 INFO:teuthology.orchestra.run.smithi031.stdout:4295 writesame 0x6a24887 thru 0x6a3266d data_size 0x1(0xdde7 bytes) 2024-09-13T07:13:19.597 INFO:teuthology.orchestra.run.smithi031.stdout:4296 read 0x9804daa thru 0x98094da (0x4731 bytes) 2024-09-13T07:13:19.597 INFO:teuthology.orchestra.run.smithi031.stdout:4297 write 0x8a3ce1 thru 0x8b2d7b (0xf09b bytes) 2024-09-13T07:13:19.599 INFO:teuthology.orchestra.run.smithi031.stdout:4298 read 0x1ca7386 thru 0x1cb661d (0xf298 bytes) 2024-09-13T07:13:19.599 INFO:teuthology.orchestra.run.smithi031.stdout:4299 write 0x1f9a443 thru 0x1fa1cad (0x786b bytes) 2024-09-13T07:13:19.602 INFO:teuthology.orchestra.run.smithi031.stdout:4300 write 0x6bdf40b thru 0x6bea9bf (0xb5b5 bytes) 2024-09-13T07:13:19.613 INFO:teuthology.orchestra.run.smithi031.stdout:4302 write 0xa55f372 thru 0xa560caf (0x193e bytes) 2024-09-13T07:13:19.624 INFO:teuthology.orchestra.run.smithi031.stdout:4304 writesame 0xa8eae3e thru 0xa8eed12 data_size 0x1(0x3ed5 bytes) 2024-09-13T07:13:19.624 INFO:teuthology.orchestra.run.smithi031.stdout:4305 read 0x47c0cb thru 0x48b9fc (0xf932 bytes) 2024-09-13T07:13:19.629 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:19.630+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:19.635 INFO:teuthology.orchestra.run.smithi031.stdout:4306 write 0xe635b45 thru 0xe64280a (0xccc6 bytes) 2024-09-13T07:13:19.642 INFO:teuthology.orchestra.run.smithi031.stdout:4307 read 0x29caf6c thru 0x29d972b (0xe7c0 bytes) 2024-09-13T07:13:19.642 INFO:teuthology.orchestra.run.smithi031.stdout:4308 punch from 0x8ae40af to 0x8ae973d, (0x568e bytes) 2024-09-13T07:13:19.642 INFO:teuthology.orchestra.run.smithi031.stdout:4310 writesame 0x9e706b0 thru 0x9e7b597 data_size 0x74(0xaee8 bytes) 2024-09-13T07:13:19.645 INFO:teuthology.orchestra.run.smithi031.stdout:4312 write 0x6c4bb3d thru 0x6c5559c (0x9a60 bytes) 2024-09-13T07:13:19.657 INFO:teuthology.orchestra.run.smithi031.stdout:4314 trunc from 0xe64280b to 0x802fc4a 2024-09-13T07:13:19.659 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:19.659 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:19.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:19.679 INFO:teuthology.orchestra.run.smithi031.stdout:4317 punch from 0x6c0e07d to 0x6c1c4e9, (0xe46c bytes) 2024-09-13T07:13:19.679 INFO:teuthology.orchestra.run.smithi031.stdout:4318 writesame 0x50f58e8 thru 0x51042f1 data_size 0x1d(0xea0a bytes) 2024-09-13T07:13:19.689 INFO:teuthology.orchestra.run.smithi031.stdout:4319 trunc from 0x802fc4a to 0xbd84b7b 2024-09-13T07:13:19.697 INFO:teuthology.orchestra.run.smithi031.stdout:4320 writesame 0x99f14f1 thru 0x99f4547 data_size 0x37(0x3057 bytes) 2024-09-13T07:13:19.701 INFO:teuthology.orchestra.run.smithi031.stdout:4321 punch from 0xb37818a to 0xb37acbb, (0x2b31 bytes) 2024-09-13T07:13:19.701 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.701 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.701 INFO:teuthology.orchestra.run.smithi031.stdout:4327 punch from 0xb02d5c1 to 0xb030fed, (0x3a2c bytes) 2024-09-13T07:13:19.701 INFO:teuthology.orchestra.run.smithi031.stdout:4330 punch from 0xa4792 to 0xaf88f, (0xb0fd bytes) 2024-09-13T07:13:19.701 INFO:teuthology.orchestra.run.smithi031.stdout:4331 writesame 0x9034511 thru 0x903936f data_size 0x1(0x4e5f bytes) 2024-09-13T07:13:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:4332 punch from 0x1cf7f7a to 0x1cfd7f2, (0x5878 bytes) 2024-09-13T07:13:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:4335 punch from 0x620f485 to 0x621e864, (0xf3df bytes) 2024-09-13T07:13:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:4336 punch from 0x5c703ea to 0x5c7d59f, (0xd1b5 bytes) 2024-09-13T07:13:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:4337 write 0xace603f thru 0xacef45a (0x941c bytes) 2024-09-13T07:13:19.715 INFO:teuthology.orchestra.run.smithi031.stdout:4339 write 0x221b5b0 thru 0x2226055 (0xaaa6 bytes) 2024-09-13T07:13:19.718 INFO:teuthology.orchestra.run.smithi031.stdout:4341 read 0x70c3d27 thru 0x70ca02b (0x6305 bytes) 2024-09-13T07:13:19.719 INFO:teuthology.orchestra.run.smithi031.stdout:4342 punch from 0x922fd03 to 0x923b41c, (0xb719 bytes) 2024-09-13T07:13:19.719 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.719 INFO:teuthology.orchestra.run.smithi031.stdout:4346 write 0x391eca thru 0x392482 (0x5b9 bytes) 2024-09-13T07:13:19.720 INFO:teuthology.orchestra.run.smithi031.stdout:4348 read 0x206ced0 thru 0x20700d4 (0x3205 bytes) 2024-09-13T07:13:19.721 INFO:teuthology.orchestra.run.smithi031.stdout:4349 punch from 0x2d5676f to 0x2d5d0d0, (0x6961 bytes) 2024-09-13T07:13:19.721 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.721 INFO:teuthology.orchestra.run.smithi031.stdout:4352 write 0x4cfcad3 thru 0x4d06e44 (0xa372 bytes) 2024-09-13T07:13:19.726 INFO:teuthology.orchestra.run.smithi031.stdout:4353 punch from 0x58264bc to 0x582871d, (0x2261 bytes) 2024-09-13T07:13:19.726 INFO:teuthology.orchestra.run.smithi031.stdout:4354 read 0x691c130 thru 0x6922160 (0x6031 bytes) 2024-09-13T07:13:19.729 INFO:teuthology.orchestra.run.smithi031.stdout:4356 trunc from 0xbd84b7b to 0x9a83cf0 2024-09-13T07:13:19.730 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:19.730+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:19.744 INFO:teuthology.orchestra.run.smithi031.stdout:4357 read 0x1055147 thru 0x105b4db (0x6395 bytes) 2024-09-13T07:13:19.745 INFO:teuthology.orchestra.run.smithi031.stdout:4358 trunc from 0x9a83cf0 to 0x87cc17c 2024-09-13T07:13:19.764 INFO:teuthology.orchestra.run.smithi031.stdout:4359 writesame 0x8a5844c thru 0x8a5fda9 data_size 0x41(0x795e bytes) 2024-09-13T07:13:19.764 INFO:teuthology.orchestra.run.smithi031.stdout:4360 trunc from 0x8a5fdaa to 0x9767480 2024-09-13T07:13:19.770 INFO:teuthology.orchestra.run.smithi031.stdout:4361 trunc from 0x9767480 to 0x96fa2b0 2024-09-13T07:13:19.781 INFO:teuthology.orchestra.run.smithi031.stdout:4362 trunc from 0x96fa2b0 to 0xe5daf6e 2024-09-13T07:13:19.788 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.788 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.788 INFO:teuthology.orchestra.run.smithi031.stdout:4365 trunc from 0xe5daf6e to 0xe1cfb3d 2024-09-13T07:13:19.793 INFO:teuthology.orchestra.run.smithi031.stdout:4366 writesame 0x8ee1446 thru 0x8ee43a5 data_size 0x10(0x2f60 bytes) 2024-09-13T07:13:19.798 INFO:teuthology.orchestra.run.smithi031.stdout:4367 read 0x6bc910b thru 0x6bd8cb4 (0xfbaa bytes) 2024-09-13T07:13:19.799 INFO:teuthology.orchestra.run.smithi031.stdout:4369 trunc from 0xe1cfb3d to 0xec6cf7e 2024-09-13T07:13:19.799 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:19.799 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:19.804 INFO:teuthology.orchestra.run.smithi031.stdout:4370 punch from 0x4ad1f23 to 0x4ad3dda, (0x1eb7 bytes) 2024-09-13T07:13:19.805 INFO:teuthology.orchestra.run.smithi031.stdout:4371 punch from 0x61377bf to 0x6146f2b, (0xf76c bytes) 2024-09-13T07:13:19.805 INFO:teuthology.orchestra.run.smithi031.stdout:4372 writesame 0xcf6b78 thru 0xcfdf80 data_size 0xd(0x7409 bytes) 2024-09-13T07:13:19.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:19.810 INFO:teuthology.orchestra.run.smithi031.stdout:4374 trunc from 0xec6cf7e to 0xd6f55a1 2024-09-13T07:13:19.815 INFO:teuthology.orchestra.run.smithi031.stdout:4375 write 0x9c1bb41 thru 0x9c2195b (0x5e1b bytes) 2024-09-13T07:13:19.818 INFO:teuthology.orchestra.run.smithi031.stdout:4376 write 0x655e375 thru 0x656e139 (0xfdc5 bytes) 2024-09-13T07:13:19.821 INFO:teuthology.orchestra.run.smithi031.stdout:4377 read 0x5a9fa49 thru 0x5aade9c (0xe454 bytes) 2024-09-13T07:13:19.832 INFO:teuthology.orchestra.run.smithi031.stdout:4378 trunc from 0xd6f55a1 to 0x5ea94cf 2024-09-13T07:13:19.833 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:19.831+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:19.844 INFO:teuthology.orchestra.run.smithi031.stdout:4379 read 0x82407f thru 0x82d49c (0x941e bytes) 2024-09-13T07:13:19.860 INFO:teuthology.orchestra.run.smithi031.stdout:4380 writesame 0xe25f5ad thru 0xe269a1e data_size 0x1f(0xa472 bytes) 2024-09-13T07:13:19.860 INFO:teuthology.orchestra.run.smithi031.stdout:4382 write 0x63d8885 thru 0x63e0fa0 (0x871c bytes) 2024-09-13T07:13:19.864 INFO:teuthology.orchestra.run.smithi031.stdout:4383 writesame 0x1537c18 thru 0x153d36a data_size 0x11(0x5753 bytes) 2024-09-13T07:13:19.867 INFO:teuthology.orchestra.run.smithi031.stdout:4384 writesame 0x403a13f thru 0x4047136 data_size 0x2c(0xcff8 bytes) 2024-09-13T07:13:19.871 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.871 INFO:teuthology.orchestra.run.smithi031.stdout:4386 write 0x38a63cd thru 0x38afc14 (0x9848 bytes) 2024-09-13T07:13:19.874 INFO:teuthology.orchestra.run.smithi031.stdout:4388 read 0x3dda283 thru 0x3de7c5e (0xd9dc bytes) 2024-09-13T07:13:19.874 INFO:teuthology.orchestra.run.smithi031.stdout:4389 read 0x4b38580 thru 0x4b43f81 (0xba02 bytes) 2024-09-13T07:13:19.889 INFO:teuthology.orchestra.run.smithi031.stdout:4391 trunc from 0xe269a1f to 0x171e72b 2024-09-13T07:13:19.916 INFO:teuthology.orchestra.run.smithi031.stdout:4392 writesame 0x88b3fbb thru 0x88c2d52 data_size 0x4(0xed98 bytes) 2024-09-13T07:13:19.923 INFO:teuthology.orchestra.run.smithi031.stdout:4393 writesame 0x9e6c223 thru 0x9e7a22a data_size 0x86(0xe008 bytes) 2024-09-13T07:13:19.923 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.923 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.923 INFO:teuthology.orchestra.run.smithi031.stdout:4397 read 0x4d3fa9e thru 0x4d46799 (0x6cfc bytes) 2024-09-13T07:13:19.923 INFO:teuthology.orchestra.run.smithi031.stdout:4399 punch from 0x655d42e to 0x656b6f2, (0xe2c4 bytes) 2024-09-13T07:13:19.924 INFO:teuthology.orchestra.run.smithi031.stdout:4400 punch from 0x7f56cf7 to 0x7f576a8, (0x9b1 bytes) 2024-09-13T07:13:19.931 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:19.932+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:19.931 INFO:teuthology.orchestra.run.smithi031.stdout:4401 trunc from 0x9e7a22b to 0x1e1127e 2024-09-13T07:13:19.937 INFO:teuthology.orchestra.run.smithi031.stdout:4403 read 0x5a5f90 thru 0x5ae765 (0x87d6 bytes) 2024-09-13T07:13:19.938 INFO:teuthology.orchestra.run.smithi031.stdout:4404 read 0xc3fa06 thru 0xc4632c (0x6927 bytes) 2024-09-13T07:13:19.938 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:19.939 INFO:teuthology.orchestra.run.smithi031.stdout:4406 writesame 0xa27ab8 thru 0xa3273a data_size 0x15(0xac83 bytes) 2024-09-13T07:13:19.941 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:19.941 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:19.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:19.941 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:19.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.942 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.942 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:19.942 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378906663, 2024-09-13T07:13:19.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:13:19.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:13:19.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.943 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:19.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:19.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:13:19.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378881349, 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:19.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378846983000001, 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:13:19.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378818159, 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:13:19.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378799861999999, 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378775056, 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378757182999999, 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:19.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:13:19.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378730842, 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:13:19.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:13:19.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:19.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.37870412, 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:19.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:13:19.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:13:19.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378686535, 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:13:19.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:19.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378659897, 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:19.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:13:19.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378642437, 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:19.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378615479, 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:19.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:13:20.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378589275, 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:13:20.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:13:20.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:13:20.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378571282999999, 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:13:20.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:20.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378479516000001, 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:13:20.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378457215999999, 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:13:20.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378426214999999, 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:13:20.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:20.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378401458999999, 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:20.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:13:20.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.378383767000001, 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:20.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:13:20.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:13:20.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:13:20.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:20.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:20.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:20.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:13:20.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:13:20.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:20.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:20.037 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:20.038 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.038 INFO:teuthology.orchestra.run.smithi031.stdout:4409 read 0xd1f6b3 thru 0xd2b129 (0xba77 bytes) 2024-09-13T07:13:20.038 INFO:teuthology.orchestra.run.smithi031.stdout:4410 read 0x157be66 thru 0x1584147 (0x82e2 bytes) 2024-09-13T07:13:20.038 INFO:teuthology.orchestra.run.smithi031.stdout:4411 punch from 0xcf6dba to 0xcf7b73, (0xdb9 bytes) 2024-09-13T07:13:20.038 INFO:teuthology.orchestra.run.smithi031.stdout:4412 write 0x4363ad8 thru 0x436d2db (0x9804 bytes) 2024-09-13T07:13:20.039 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4413 writesame 0x744e53d thru 0x745da32 data_size 0x5(0xf4f6 bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4415 punch from 0x4ed2963 to 0x4edd867, (0xaf04 bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4416 punch from 0x478fa73 to 0x4792755, (0x2ce2 bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4417 writesame 0x731ecff thru 0x73292a2 data_size 0x2(0xa5a4 bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4418 read 0x6bab9f1 thru 0x6bb0fad (0x55bd bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4420 punch from 0x2e21c67 to 0x2e2a9e0, (0x8d79 bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4421 writesame 0x8b7d3cd thru 0x8b8071d data_size 0x1d(0x3351 bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4422 read 0x253fd28 thru 0x254a319 (0xa5f2 bytes) 2024-09-13T07:13:20.042 INFO:teuthology.orchestra.run.smithi031.stdout:4423 write 0xb411a01 thru 0xb42088a (0xee8a bytes) 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:4424 trunc from 0xb42088b to 0x3237455 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:4428 read 0x1c313ec thru 0x1c39716 (0x832b bytes) 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:4429 punch from 0xf2da99 to 0xf322c3, (0x482a bytes) 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:4430 writesame 0xa8dcb9f thru 0xa8eadb6 data_size 0x14(0xe218 bytes) 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:4431 trunc from 0xa8eadb7 to 0x7141094 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:4432 write 0x20f016d thru 0x20f1247 (0x10db bytes) 2024-09-13T07:13:20.043 INFO:teuthology.orchestra.run.smithi031.stdout:4433 trunc from 0x7141094 to 0x5c4f320 2024-09-13T07:13:20.044 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.044 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.044 INFO:teuthology.orchestra.run.smithi031.stdout:4436 read 0x15247c1 thru 0x1525408 (0xc48 bytes) 2024-09-13T07:13:20.044 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:20.044+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:20.045 INFO:teuthology.orchestra.run.smithi031.stdout:4437 punch from 0x23487c6 to 0x234c730, (0x3f6a bytes) 2024-09-13T07:13:20.045 INFO:teuthology.orchestra.run.smithi031.stdout:4439 write 0xb12779 thru 0xb18508 (0x5d90 bytes) 2024-09-13T07:13:20.045 INFO:teuthology.orchestra.run.smithi031.stdout:4440 punch from 0x45ccbe to 0x45f0ad, (0x23ef bytes) 2024-09-13T07:13:20.045 INFO:teuthology.orchestra.run.smithi031.stdout:4442 punch from 0x373f1f5 to 0x3743a02, (0x480d bytes) 2024-09-13T07:13:20.045 INFO:teuthology.orchestra.run.smithi031.stdout:4443 read 0x38e9945 thru 0x38f84a9 (0xeb65 bytes) 2024-09-13T07:13:20.046 INFO:teuthology.orchestra.run.smithi031.stdout:4444 write 0x31cbf09 thru 0x31d7dbb (0xbeb3 bytes) 2024-09-13T07:13:20.049 INFO:teuthology.orchestra.run.smithi031.stdout:4445 write 0x21b0c53 thru 0x21bffbe (0xf36c bytes) 2024-09-13T07:13:20.052 INFO:teuthology.orchestra.run.smithi031.stdout:4446 read 0x5118df1 thru 0x511f03e (0x624e bytes) 2024-09-13T07:13:20.052 INFO:teuthology.orchestra.run.smithi031.stdout:4448 punch from 0x363e86b to 0x363fc4a, (0x13df bytes) 2024-09-13T07:13:20.058 INFO:teuthology.orchestra.run.smithi031.stdout:4449 write 0x6729204 thru 0x672951f (0x31c bytes) 2024-09-13T07:13:20.061 INFO:teuthology.orchestra.run.smithi031.stdout:4450 write 0x3757a40 thru 0x37620ee (0xa6af bytes) 2024-09-13T07:13:20.064 INFO:teuthology.orchestra.run.smithi031.stdout:4452 punch from 0x2f2e626 to 0x2f2e81f, (0x1f9 bytes) 2024-09-13T07:13:20.071 INFO:teuthology.orchestra.run.smithi031.stdout:4453 write 0x799ca22 thru 0x79a3b7e (0x715d bytes) 2024-09-13T07:13:20.074 INFO:teuthology.orchestra.run.smithi031.stdout:4454 read 0x1ce44d3 thru 0x1cedbd4 (0x9702 bytes) 2024-09-13T07:13:20.074 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.081 INFO:teuthology.orchestra.run.smithi031.stdout:4457 write 0x906e011 thru 0x90740e4 (0x60d4 bytes) 2024-09-13T07:13:20.096 INFO:teuthology.orchestra.run.smithi031.stdout:4459 writesame 0xd4f6060 thru 0xd4fae99 data_size 0x26(0x4e3a bytes) 2024-09-13T07:13:20.096 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.096 INFO:teuthology.orchestra.run.smithi031.stdout:4463 write 0xd0ce660 thru 0xd0d3096 (0x4a37 bytes) 2024-09-13T07:13:20.099 INFO:teuthology.orchestra.run.smithi031.stdout:4464 write 0x104a3b5 thru 0x1054c06 (0xa852 bytes) 2024-09-13T07:13:20.108 INFO:teuthology.orchestra.run.smithi031.stdout:4465 writesame 0xdcfd747 thru 0xdd065f3 data_size 0x19(0x8ead bytes) 2024-09-13T07:13:20.108 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.108 INFO:teuthology.orchestra.run.smithi031.stdout:4468 write 0xfdcdff thru 0xfe4404 (0x7606 bytes) 2024-09-13T07:13:20.109 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.109 INFO:teuthology.orchestra.run.smithi031.stdout:4470 write 0x6fb6a5a thru 0x6fbff13 (0x94ba bytes) 2024-09-13T07:13:20.113 INFO:teuthology.orchestra.run.smithi031.stdout:4471 writesame 0xc73af2 thru 0xc7f734 data_size 0x3f(0xbc43 bytes) 2024-09-13T07:13:20.121 INFO:teuthology.orchestra.run.smithi031.stdout:4474 write 0x8743703 thru 0x874a160 (0x6a5e bytes) 2024-09-13T07:13:20.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:20.127 INFO:teuthology.orchestra.run.smithi031.stdout:4475 punch from 0x6f28a09 to 0x6f2bc90, (0x3287 bytes) 2024-09-13T07:13:20.127 INFO:teuthology.orchestra.run.smithi031.stdout:4476 write 0x59b39b3 thru 0x59b7b0d (0x415b bytes) 2024-09-13T07:13:20.131 INFO:teuthology.orchestra.run.smithi031.stdout:4477 trunc from 0xdd065f4 to 0xca9ea64 2024-09-13T07:13:20.141 INFO:teuthology.orchestra.run.smithi031.stdout:4478 writesame 0x8e86c2f thru 0x8e94b94 data_size 0xe(0xdf66 bytes) 2024-09-13T07:13:20.143 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:20.144+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:20.145 INFO:teuthology.orchestra.run.smithi031.stdout:4479 write 0xbaf3fb1 thru 0xbb026d8 (0xe728 bytes) 2024-09-13T07:13:20.149 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:20.149 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:20.149 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.149 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:86233 3.9 3:96e8c531:::rbd_data.11732319a2dc.00000000000000bd:head [set-alloc-hint object_size 1048576 write_size 1048576,write 212913~49231 in=49231b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:13:20.149 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:20.148786+0000", 2024-09-13T07:13:20.149 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0022586659999999999, 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022776609999999998, 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 86233 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148786+0000", 2024-09-13T07:13:20.150 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148786+0000", 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148786+0000", 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.151 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148836+0000", 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0918999999999998e-05 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148849+0000", 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2459e-05 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148856+0000", 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1289999999999998e-06 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.152 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148900+0000", 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4159000000000002e-05 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.148962+0000", 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2222000000000002e-05 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 2", 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149048+0000", 2024-09-13T07:13:20.153 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5218999999999998e-05 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.150135+0000", 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001087408 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:86234 3.13 e58/56 3:cd309d41:::rbd_data.11732319a2dc.00000000000000be:head v 58'371, mlcod=58'369)", 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:20.149575+0000", 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0014702560000000001, 2024-09-13T07:13:20.154 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015364230000000001, 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149575+0000", 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149575+0000", 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.155 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149574+0000", 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149587+0000", 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.296e-05 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149598+0000", 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1274000000000001e-05 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.156 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149605+0000", 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5710000000000003e-06 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149634+0000", 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9462999999999999e-05 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:20.149648+0000", 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3298e-05 2024-09-13T07:13:20.157 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:20.158 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:20.158 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:20.158 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:20.158 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-13T07:13:20.158 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 2 2024-09-13T07:13:20.158 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:20.159 INFO:teuthology.orchestra.run.smithi031.stdout:4480 punch from 0x9223627 to 0x92272f4, (0x3ccd bytes) 2024-09-13T07:13:20.159 INFO:teuthology.orchestra.run.smithi031.stdout:4482 write 0x6859991 thru 0x6863277 (0x98e7 bytes) 2024-09-13T07:13:20.159 INFO:teuthology.orchestra.run.smithi031.stdout:4483 write 0x4a070ad thru 0x4a1697b (0xf8cf bytes) 2024-09-13T07:13:20.159 INFO:teuthology.orchestra.run.smithi031.stdout:4484 write 0x9dcc953 thru 0x9ddbad6 (0xf184 bytes) 2024-09-13T07:13:20.160 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:4485 punch from 0xb93f2ba to 0xb94de39, (0xeb7f bytes) 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:4486 punch from 0x5dd14c6 to 0x5ddf6bd, (0xe1f7 bytes) 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:4489 read 0x1a7f359 thru 0x1a84438 (0x50e0 bytes) 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:4490 read 0x2620fbe thru 0x26263f0 (0x5433 bytes) 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:4491 read 0xbd61ba9 thru 0xbd6f7ec (0xdc44 bytes) 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:4492 punch from 0x53dbd61 to 0x53e9088, (0xd327 bytes) 2024-09-13T07:13:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:4493 write 0xa20a082 thru 0xa21679d (0xc71c bytes) 2024-09-13T07:13:20.164 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.178 INFO:teuthology.orchestra.run.smithi031.stdout:4495 trunc from 0xca9ea64 to 0x1541b2 2024-09-13T07:13:20.212 INFO:teuthology.orchestra.run.smithi031.stdout:4496 writesame 0x5ff04d3 thru 0x5ff65ef data_size 0x1(0x611d bytes) 2024-09-13T07:13:20.213 INFO:teuthology.orchestra.run.smithi031.stdout:4498 trunc from 0x5ff65f0 to 0x64c3976 2024-09-13T07:13:20.218 INFO:teuthology.orchestra.run.smithi031.stdout:4499 read 0x1e9ad28 thru 0x1ea8ca1 (0xdf7a bytes) 2024-09-13T07:13:20.218 INFO:teuthology.orchestra.run.smithi031.stdout:4500 read 0x1948360 thru 0x19515f6 (0x9297 bytes) 2024-09-13T07:13:20.218 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.218 INFO:teuthology.orchestra.run.smithi031.stdout:4503 read 0x3d5d4bb thru 0x3d6826f (0xadb5 bytes) 2024-09-13T07:13:20.231 INFO:teuthology.orchestra.run.smithi031.stdout:4505 write 0xd8d4974 thru 0xd8db8e0 (0x6f6d bytes) 2024-09-13T07:13:20.234 INFO:teuthology.orchestra.run.smithi031.stdout:4506 read 0x852b9dd thru 0x8539df7 (0xe41b bytes) 2024-09-13T07:13:20.242 INFO:teuthology.orchestra.run.smithi031.stdout:4507 write 0xdb2c33c thru 0xdb35f7d (0x9c42 bytes) 2024-09-13T07:13:20.244 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:20.245+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:20.244 INFO:teuthology.orchestra.run.smithi031.stdout:4508 writesame 0x606bc0a thru 0x606dfa6 data_size 0x3(0x239d bytes) 2024-09-13T07:13:20.249 INFO:teuthology.orchestra.run.smithi031.stdout:4509 punch from 0x81288d1 to 0x8133af9, (0xb228 bytes) 2024-09-13T07:13:20.249 INFO:teuthology.orchestra.run.smithi031.stdout:4510 punch from 0x6c84652 to 0x6c90df2, (0xc7a0 bytes) 2024-09-13T07:13:20.249 INFO:teuthology.orchestra.run.smithi031.stdout:4511 write 0x31ebb3d thru 0x31f15c6 (0x5a8a bytes) 2024-09-13T07:13:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:4514 punch from 0xd247856 to 0xd253155, (0xb8ff bytes) 2024-09-13T07:13:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:4515 write 0x24e76a3 thru 0x24f5369 (0xdcc7 bytes) 2024-09-13T07:13:20.256 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.262 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:20.262 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:20.265 INFO:teuthology.orchestra.run.smithi031.stdout:4517 trunc from 0xdb35f7e to 0x576658a 2024-09-13T07:13:20.268 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:20.269 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:20.269 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:20.269 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:20.269 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:20.275 INFO:teuthology.orchestra.run.smithi031.stdout:4518 write 0x4dc155 thru 0x4e4df2 (0x8c9e bytes) 2024-09-13T07:13:20.277 INFO:teuthology.orchestra.run.smithi031.stdout:4519 writesame 0x57012a8 thru 0x570efb3 data_size 0x2(0xdd0c bytes) 2024-09-13T07:13:20.278 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:20.282 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:20.282 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:20.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:20.299 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:20.302 INFO:teuthology.orchestra.run.smithi031.stdout:4520 writesame 0x845e1d3 thru 0x845ffeb data_size 0x17(0x1e19 bytes) 2024-09-13T07:13:20.302 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.302 INFO:teuthology.orchestra.run.smithi031.stdout:4526 punch from 0x399b7fc to 0x39a2fbe, (0x77c2 bytes) 2024-09-13T07:13:20.304 INFO:teuthology.orchestra.run.smithi031.stdout:4527 trunc from 0x845ffec to 0xa040015 2024-09-13T07:13:20.309 INFO:teuthology.orchestra.run.smithi031.stdout:4528 writesame 0x6f8beb5 thru 0x6f97c86 data_size 0x4e(0xbdd2 bytes) 2024-09-13T07:13:20.313 INFO:teuthology.orchestra.run.smithi031.stdout:4530 read 0x72c063 thru 0x731b2f (0x5acd bytes) 2024-09-13T07:13:20.317 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:20.322 INFO:teuthology.orchestra.run.smithi031.stdout:4532 trunc from 0xa040015 to 0x28f9ee5 2024-09-13T07:13:20.332 INFO:teuthology.orchestra.run.smithi031.stdout:4533 read 0x1738fe1 thru 0x173ad01 (0x1d21 bytes) 2024-09-13T07:13:20.341 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:20.341 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:20.345 INFO:teuthology.orchestra.run.smithi031.stdout:4534 write 0x8fd4bac thru 0x8fda17b (0x55d0 bytes) 2024-09-13T07:13:20.349 INFO:teuthology.orchestra.run.smithi031.stdout:4535 write 0xda078c thru 0xda7da8 (0x761d bytes) 2024-09-13T07:13:20.360 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:20.360 INFO:teuthology.orchestra.run.smithi031.stdout:4536 writesame 0xb9cabbb thru 0xb9cd918 data_size 0x2(0x2d5e bytes) 2024-09-13T07:13:20.361 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.361 INFO:teuthology.orchestra.run.smithi031.stdout:4540 punch from 0xb17367c to 0xb17a03d, (0x69c1 bytes) 2024-09-13T07:13:20.361 INFO:teuthology.orchestra.run.smithi031.stdout:4541 write 0x358b86e thru 0x3599603 (0xdd96 bytes) 2024-09-13T07:13:20.362 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:20.363+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:20.364 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.364 INFO:teuthology.orchestra.run.smithi031.stdout:4544 write 0x3be34cc thru 0x3be3c9f (0x7d4 bytes) 2024-09-13T07:13:20.376 INFO:teuthology.orchestra.run.smithi031.stdout:4545 trunc from 0xb9cd919 to 0x32bf3e9 2024-09-13T07:13:20.377 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:13:20.399 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:20.400 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:20.401 INFO:teuthology.orchestra.run.smithi031.stdout:4546 write 0x60826cf thru 0x608a716 (0x8048 bytes) 2024-09-13T07:13:20.405 INFO:teuthology.orchestra.run.smithi031.stdout:4547 punch from 0x4d09859 to 0x4d0bb11, (0x22b8 bytes) 2024-09-13T07:13:20.405 INFO:teuthology.orchestra.run.smithi031.stdout:4548 write 0xd1ec99 thru 0xd2093f (0x1ca7 bytes) 2024-09-13T07:13:20.415 INFO:teuthology.orchestra.run.smithi031.stdout:4549 write 0x7f778f7 thru 0x7f7ce0f (0x5519 bytes) 2024-09-13T07:13:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:4550 punch from 0x483c229 to 0x4840a3f, (0x4816 bytes) 2024-09-13T07:13:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:4551 write 0x11f56ab thru 0x11fb8c9 (0x621f bytes) 2024-09-13T07:13:20.420 INFO:teuthology.orchestra.run.smithi031.stdout:4553 read 0x56f3a4a thru 0x56fd131 (0x96e8 bytes) 2024-09-13T07:13:20.426 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:20.426 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:4554 write 0x8b7a643 thru 0x8b861f1 (0xbbaf bytes) 2024-09-13T07:13:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:4555 write 0x222725 thru 0x224bc0 (0x249c bytes) 2024-09-13T07:13:20.433 INFO:teuthology.orchestra.run.smithi031.stdout:4556 write 0x438c9f3 thru 0x43954e8 (0x8af6 bytes) 2024-09-13T07:13:20.435 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:20.438 INFO:teuthology.orchestra.run.smithi031.stdout:4557 trunc from 0x8b861f2 to 0x7ff1433 2024-09-13T07:13:20.450 INFO:teuthology.orchestra.run.smithi031.stdout:4560 write 0x10a632f thru 0x10b5244 (0xef16 bytes) 2024-09-13T07:13:20.459 INFO:teuthology.orchestra.run.smithi031.stdout:4562 write 0xabc56e3 thru 0xabd2d95 (0xd6b3 bytes) 2024-09-13T07:13:20.464 INFO:teuthology.orchestra.run.smithi031.stdout:4563 write 0x5f140ee thru 0x5f19838 (0x574b bytes) 2024-09-13T07:13:20.467 INFO:teuthology.orchestra.run.smithi031.stdout:4564 write 0x37cd97c thru 0x37d252f (0x4bb4 bytes) 2024-09-13T07:13:20.470 INFO:teuthology.orchestra.run.smithi031.stdout:4567 writesame 0x95ac292 thru 0x95b0148 data_size 0x13(0x3eb7 bytes) 2024-09-13T07:13:20.473 INFO:teuthology.orchestra.run.smithi031.stdout:4568 read 0x9b2508c thru 0x9b2c97d (0x78f2 bytes) 2024-09-13T07:13:20.477 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:20.478+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:20.478 INFO:teuthology.orchestra.run.smithi031.stdout:4569 trunc from 0xabd2d96 to 0x6340f58 2024-09-13T07:13:20.489 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.489 INFO:teuthology.orchestra.run.smithi031.stdout:4571 read 0x6aade thru 0x6d785 (0x2ca8 bytes) 2024-09-13T07:13:20.498 INFO:teuthology.orchestra.run.smithi031.stdout:4573 trunc from 0x6340f58 to 0xd4ade1a 2024-09-13T07:13:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:4574 trunc from 0xd4ade1a to 0xc9c7762 2024-09-13T07:13:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:4575 read 0x5169d6e thru 0x516d168 (0x33fb bytes) 2024-09-13T07:13:20.517 INFO:teuthology.orchestra.run.smithi031.stdout:4577 write 0xd61afef thru 0xd61ce0d (0x1e1f bytes) 2024-09-13T07:13:20.520 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.520 INFO:teuthology.orchestra.run.smithi031.stdout:4580 writesame 0x8298773 thru 0x8299e41 data_size 0x1(0x16cf bytes) 2024-09-13T07:13:20.524 INFO:teuthology.orchestra.run.smithi031.stdout:4581 read 0x3272050 thru 0x3278363 (0x6314 bytes) 2024-09-13T07:13:20.524 INFO:teuthology.orchestra.run.smithi031.stdout:4582 punch from 0x1dfb100 to 0x1dfec7f, (0x3b7f bytes) 2024-09-13T07:13:20.530 INFO:teuthology.orchestra.run.smithi031.stdout:4583 trunc from 0xd61ce0e to 0x80b7272 2024-09-13T07:13:20.541 INFO:teuthology.orchestra.run.smithi031.stdout:4585 read 0x537a74d thru 0x5386fa5 (0xc859 bytes) 2024-09-13T07:13:20.541 INFO:teuthology.orchestra.run.smithi031.stdout:4586 writesame 0x1468c40 thru 0x1469751 data_size 0xd(0xb12 bytes) 2024-09-13T07:13:20.544 INFO:teuthology.orchestra.run.smithi031.stdout:4587 writesame 0x579bcea thru 0x57a782e data_size 0x1(0xbb45 bytes) 2024-09-13T07:13:20.546 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:20.547+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:20.569 INFO:teuthology.orchestra.run.smithi031.stdout:4589 punch from 0x32ac0a5 to 0x32bc09c, (0xfff7 bytes) 2024-09-13T07:13:20.569 INFO:teuthology.orchestra.run.smithi031.stdout:4591 punch from 0x10cf623 to 0x10de148, (0xeb25 bytes) 2024-09-13T07:13:20.569 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.569 INFO:teuthology.orchestra.run.smithi031.stdout:4593 read 0x3928489 thru 0x392ab19 (0x2691 bytes) 2024-09-13T07:13:20.580 INFO:teuthology.orchestra.run.smithi031.stdout:4594 write 0xc62439c thru 0xc62b40a (0x706f bytes) 2024-09-13T07:13:20.583 INFO:teuthology.orchestra.run.smithi031.stdout:4595 write 0xa012401 thru 0xa01e02c (0xbc2c bytes) 2024-09-13T07:13:20.584 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:20.584 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:20.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:20.594 INFO:teuthology.orchestra.run.smithi031.stdout:4596 trunc from 0xc62b40b to 0x4f3bb66 2024-09-13T07:13:20.604 INFO:teuthology.orchestra.run.smithi031.stdout:4597 read 0xc9919f thru 0xc9cd8f (0x3bf1 bytes) 2024-09-13T07:13:20.605 INFO:teuthology.orchestra.run.smithi031.stdout:4598 punch from 0x16ff12a to 0x170c556, (0xd42c bytes) 2024-09-13T07:13:20.605 INFO:teuthology.orchestra.run.smithi031.stdout:4599 read 0x1d687c2 thru 0x1d6b2e3 (0x2b22 bytes) 2024-09-13T07:13:20.611 INFO:teuthology.orchestra.run.smithi031.stdout:4601 trunc from 0x4f3bb66 to 0xabebc64 2024-09-13T07:13:20.621 INFO:teuthology.orchestra.run.smithi031.stdout:4602 writesame 0xae6f778 thru 0xae7b090 data_size 0x51(0xb919 bytes) 2024-09-13T07:13:20.621 INFO:teuthology.orchestra.run.smithi031.stdout:4606 write 0x20ecda1 thru 0x20fb410 (0xe670 bytes) 2024-09-13T07:13:20.624 INFO:teuthology.orchestra.run.smithi031.stdout:4608 writesame 0xaa384a8 thru 0xaa3b53f data_size 0x14(0x3098 bytes) 2024-09-13T07:13:20.629 INFO:teuthology.orchestra.run.smithi031.stdout:4609 punch from 0x1ca456d to 0x1cb255b, (0xdfee bytes) 2024-09-13T07:13:20.629 INFO:teuthology.orchestra.run.smithi031.stdout:4610 write 0x51ccc0c thru 0x51d75a0 (0xa995 bytes) 2024-09-13T07:13:20.633 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.633 INFO:teuthology.orchestra.run.smithi031.stdout:4614 punch from 0x5ed46bc to 0x5ee16a4, (0xcfe8 bytes) 2024-09-13T07:13:20.633 INFO:teuthology.orchestra.run.smithi031.stdout:4616 writesame 0x6dbc040 thru 0x6dcb3ed data_size 0x25(0xf3ae bytes) 2024-09-13T07:13:20.640 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.640 INFO:teuthology.orchestra.run.smithi031.stdout:4620 write 0x9705275 thru 0x971387b (0xe607 bytes) 2024-09-13T07:13:20.643 INFO:teuthology.orchestra.run.smithi031.stdout:4621 read 0xac80b21 thru 0xac9080c (0xfcec bytes) 2024-09-13T07:13:20.643 INFO:teuthology.orchestra.run.smithi031.stdout:4622 read 0xa59c1e7 thru 0xa5a767c (0xb496 bytes) 2024-09-13T07:13:20.643 INFO:teuthology.orchestra.run.smithi031.stdout:4623 read 0x75e4942 thru 0x75ea641 (0x5d00 bytes) 2024-09-13T07:13:20.644 INFO:teuthology.orchestra.run.smithi031.stdout:4624 read 0x700f5ab thru 0x701f566 (0xffbc bytes) 2024-09-13T07:13:20.648 INFO:teuthology.orchestra.run.smithi031.stdout:4626 trunc from 0xae7b091 to 0x6f1f1bb 2024-09-13T07:13:20.665 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.665 INFO:teuthology.orchestra.run.smithi031.stdout:4628 read 0x3d3c10b thru 0x3d3f572 (0x3468 bytes) 2024-09-13T07:13:20.665 INFO:teuthology.orchestra.run.smithi031.stdout:4629 write 0x4b4ae5a thru 0x4b4eec4 (0x406b bytes) 2024-09-13T07:13:20.672 INFO:teuthology.orchestra.run.smithi031.stdout:4630 writesame 0x6fede4c thru 0x6ff1abf data_size 0x4(0x3c74 bytes) 2024-09-13T07:13:20.673 INFO:teuthology.orchestra.run.smithi031.stdout:4631 read 0xe59c1a thru 0xe689d2 (0xedb9 bytes) 2024-09-13T07:13:20.673 INFO:teuthology.orchestra.run.smithi031.stdout:4632 punch from 0x666e45f to 0x667ab18, (0xc6b9 bytes) 2024-09-13T07:13:20.673 INFO:teuthology.orchestra.run.smithi031.stdout:4633 read 0x1c2d2fc thru 0x1c3ca26 (0xf72b bytes) 2024-09-13T07:13:20.678 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:20.679+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:20.685 INFO:teuthology.orchestra.run.smithi031.stdout:4634 writesame 0xd894fc1 thru 0xd89d1d1 data_size 0xb(0x8211 bytes) 2024-09-13T07:13:20.685 INFO:teuthology.orchestra.run.smithi031.stdout:4635 writesame 0xbd7c380 thru 0xbd83d54 data_size 0x1(0x79d5 bytes) 2024-09-13T07:13:20.707 INFO:teuthology.orchestra.run.smithi031.stdout:4636 trunc from 0xd89d1d2 to 0x3fb0390 2024-09-13T07:13:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:4638 trunc from 0x3fb0390 to 0x57fa41f 2024-09-13T07:13:20.726 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.726 INFO:teuthology.orchestra.run.smithi031.stdout:4640 punch from 0xb5b824 to 0xb61354, (0x5b30 bytes) 2024-09-13T07:13:20.731 INFO:teuthology.orchestra.run.smithi031.stdout:4642 trunc from 0x57fa41f to 0x625efc 2024-09-13T07:13:20.742 INFO:teuthology.orchestra.run.smithi031.stdout:4643 read 0x3cc805 thru 0x3d952f (0xcd2b bytes) 2024-09-13T07:13:20.742 INFO:teuthology.orchestra.run.smithi031.stdout:4646 read 0x3d3f41 thru 0x3d560f (0x16cf bytes) 2024-09-13T07:13:20.742 INFO:teuthology.orchestra.run.smithi031.stdout:4647 read 0x923f6 thru 0x9ac95 (0x88a0 bytes) 2024-09-13T07:13:20.743 INFO:teuthology.orchestra.run.smithi031.stdout:4648 read 0x546851 thru 0x54786d (0x101d bytes) 2024-09-13T07:13:20.743 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:20.743 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:20.747 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:20.748+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:20.752 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:20.763 INFO:teuthology.orchestra.run.smithi031.stdout:4649 write 0xd754f1e thru 0xd75e478 (0x955b bytes) 2024-09-13T07:13:20.766 INFO:teuthology.orchestra.run.smithi031.stdout:4650 write 0x25c368b thru 0x25d19ad (0xe323 bytes) 2024-09-13T07:13:20.770 INFO:teuthology.orchestra.run.smithi031.stdout:4651 read 0x1391908 thru 0x1394083 (0x277c bytes) 2024-09-13T07:13:20.770 INFO:teuthology.orchestra.run.smithi031.stdout:4652 writesame 0xb534dfe thru 0xb537351 data_size 0x2(0x2554 bytes) 2024-09-13T07:13:20.774 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.774 INFO:teuthology.orchestra.run.smithi031.stdout:4656 read 0xa97cd02 thru 0xa982529 (0x5828 bytes) 2024-09-13T07:13:20.774 INFO:teuthology.orchestra.run.smithi031.stdout:4657 read 0x1428f43 thru 0x1433f2a (0xafe8 bytes) 2024-09-13T07:13:20.774 INFO:teuthology.orchestra.run.smithi031.stdout:4658 write 0x6d6ef7b thru 0x6d77b39 (0x8bbf bytes) 2024-09-13T07:13:20.781 INFO:teuthology.orchestra.run.smithi031.stdout:4659 write 0xd9b62cd thru 0xd9ba32f (0x4063 bytes) 2024-09-13T07:13:20.785 INFO:teuthology.orchestra.run.smithi031.stdout:4661 punch from 0x6a35268 to 0x6a38186, (0x2f1e bytes) 2024-09-13T07:13:20.785 INFO:teuthology.orchestra.run.smithi031.stdout:4662 writesame 0x2306776 thru 0x230c1a4 data_size 0x1(0x5a2f bytes) 2024-09-13T07:13:20.797 INFO:teuthology.orchestra.run.smithi031.stdout:4663 read 0x37457f2 thru 0x374ba74 (0x6283 bytes) 2024-09-13T07:13:20.798 INFO:teuthology.orchestra.run.smithi031.stdout:4664 punch from 0x547f314 to 0x5481ac4, (0x27b0 bytes) 2024-09-13T07:13:20.798 INFO:teuthology.orchestra.run.smithi031.stdout:4665 write 0xd79318a thru 0xd7a09c8 (0xd83f bytes) 2024-09-13T07:13:20.800 INFO:teuthology.orchestra.run.smithi031.stdout:4666 write 0x175793 thru 0x17ac07 (0x5475 bytes) 2024-09-13T07:13:20.802 INFO:teuthology.orchestra.run.smithi031.stdout:4668 write 0x5bc96d4 thru 0x5bd2ce1 (0x960e bytes) 2024-09-13T07:13:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:4670 trunc from 0xd9ba330 to 0x485afb9 2024-09-13T07:13:20.836 INFO:teuthology.orchestra.run.smithi031.stdout:4673 write 0x9ae068b thru 0x9ae1c21 (0x1597 bytes) 2024-09-13T07:13:20.839 INFO:teuthology.orchestra.run.smithi031.stdout:4675 writesame 0x3a69c04 thru 0x3a77fba data_size 0x59(0xe3b7 bytes) 2024-09-13T07:13:20.842 INFO:teuthology.orchestra.run.smithi031.stdout:4676 write 0x5b08822 thru 0x5b119a5 (0x9184 bytes) 2024-09-13T07:13:20.848 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:20.849+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:20.856 INFO:teuthology.orchestra.run.smithi031.stdout:4677 write 0xe2c7b24 thru 0xe2c81d3 (0x6b0 bytes) 2024-09-13T07:13:20.858 INFO:teuthology.orchestra.run.smithi031.stdout:4678 punch from 0x3efc491 to 0x3f07217, (0xad86 bytes) 2024-09-13T07:13:20.858 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.858 INFO:teuthology.orchestra.run.smithi031.stdout:4680 read 0x7ae8462 thru 0x7af1313 (0x8eb2 bytes) 2024-09-13T07:13:20.858 INFO:teuthology.orchestra.run.smithi031.stdout:4681 write 0xb24065d thru 0xb24e68d (0xe031 bytes) 2024-09-13T07:13:20.861 INFO:teuthology.orchestra.run.smithi031.stdout:4682 write 0x908de5f thru 0x90993f2 (0xb594 bytes) 2024-09-13T07:13:20.865 INFO:teuthology.orchestra.run.smithi031.stdout:4683 read 0x62f9e16 thru 0x6302c3f (0x8e2a bytes) 2024-09-13T07:13:20.865 INFO:teuthology.orchestra.run.smithi031.stdout:4684 punch from 0xb7093af to 0xb7169dc, (0xd62d bytes) 2024-09-13T07:13:20.865 INFO:teuthology.orchestra.run.smithi031.stdout:4685 read 0x4074df9 thru 0x407dd10 (0x8f18 bytes) 2024-09-13T07:13:20.865 INFO:teuthology.orchestra.run.smithi031.stdout:4687 punch from 0x83f846a to 0x83fb3ce, (0x2f64 bytes) 2024-09-13T07:13:20.865 INFO:teuthology.orchestra.run.smithi031.stdout:4689 read 0xc80cf0a thru 0xc81bced (0xede4 bytes) 2024-09-13T07:13:20.866 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.866 INFO:teuthology.orchestra.run.smithi031.stdout:4693 read 0xbcfe8e6 thru 0xbd08082 (0x979d bytes) 2024-09-13T07:13:20.866 INFO:teuthology.orchestra.run.smithi031.stdout:4694 write 0x438b4bc thru 0x4390946 (0x548b bytes) 2024-09-13T07:13:20.868 INFO:teuthology.orchestra.run.smithi031.stdout:4695 read 0x5d9da61 thru 0x5d9ea54 (0xff4 bytes) 2024-09-13T07:13:20.868 INFO:teuthology.orchestra.run.smithi031.stdout:4696 write 0xd49494e thru 0xd49f236 (0xa8e9 bytes) 2024-09-13T07:13:20.871 INFO:teuthology.orchestra.run.smithi031.stdout:4697 read 0xa613270 thru 0xa6161f9 (0x2f8a bytes) 2024-09-13T07:13:20.871 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.871 INFO:teuthology.orchestra.run.smithi031.stdout:4701 read 0x35f2a03 thru 0x3601fff (0xf5fd bytes) 2024-09-13T07:13:20.871 INFO:teuthology.orchestra.run.smithi031.stdout:4703 punch from 0x8fd7ebf to 0x8fdd93d, (0x5a7e bytes) 2024-09-13T07:13:20.871 INFO:teuthology.orchestra.run.smithi031.stdout:4706 writesame 0x2932aa2 thru 0x293ae31 data_size 0x28(0x8390 bytes) 2024-09-13T07:13:20.874 INFO:teuthology.orchestra.run.smithi031.stdout:4707 writesame 0x1ae11f1 thru 0x1ae5209 data_size 0x1(0x4019 bytes) 2024-09-13T07:13:20.884 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.884 INFO:teuthology.orchestra.run.smithi031.stdout:4709 punch from 0xda51ad1 to 0xda57ab4, (0x5fe3 bytes) 2024-09-13T07:13:20.885 INFO:teuthology.orchestra.run.smithi031.stdout:4711 write 0xdd94ca8 thru 0xdd9e473 (0x97cc bytes) 2024-09-13T07:13:20.892 INFO:teuthology.orchestra.run.smithi031.stdout:4712 trunc from 0xe2c81d4 to 0xa700c78 2024-09-13T07:13:20.901 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:20.901 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:20.902 INFO:teuthology.orchestra.run.smithi031.stdout:4713 write 0x3078ff9 thru 0x3082dec (0x9df4 bytes) 2024-09-13T07:13:20.910 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:20.916 INFO:teuthology.orchestra.run.smithi031.stdout:4719 writesame 0xe33ea8d thru 0xe34d29a data_size 0x3(0xe80e bytes) 2024-09-13T07:13:20.916 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.916 INFO:teuthology.orchestra.run.smithi031.stdout:4721 write 0x827e430 thru 0x827edfc (0x9cd bytes) 2024-09-13T07:13:20.918 INFO:teuthology.orchestra.run.smithi031.stdout:4723 write 0x95afd20 thru 0x95b0455 (0x736 bytes) 2024-09-13T07:13:20.922 INFO:teuthology.orchestra.run.smithi031.stdout:4724 writesame 0x5c2ef2d thru 0x5c3184b data_size 0x21(0x291f bytes) 2024-09-13T07:13:20.927 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.927 INFO:teuthology.orchestra.run.smithi031.stdout:4728 write 0x861215 thru 0x862af2 (0x18de bytes) 2024-09-13T07:13:20.930 INFO:teuthology.orchestra.run.smithi031.stdout:4729 punch from 0x8042e1c to 0x804d305, (0xa4e9 bytes) 2024-09-13T07:13:20.930 INFO:teuthology.orchestra.run.smithi031.stdout:4730 write 0x32bd85d thru 0x32c5e7f (0x8623 bytes) 2024-09-13T07:13:20.934 INFO:teuthology.orchestra.run.smithi031.stdout:4731 write 0xad04db2 thru 0xad095b6 (0x4805 bytes) 2024-09-13T07:13:20.937 INFO:teuthology.orchestra.run.smithi031.stdout:4732 writesame 0x7d6aa05 thru 0x7d7919e data_size 0x9a(0xe79a bytes) 2024-09-13T07:13:20.941 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.941 INFO:teuthology.orchestra.run.smithi031.stdout:4735 read 0xad1751 thru 0xad6c2d (0x54dd bytes) 2024-09-13T07:13:20.941 INFO:teuthology.orchestra.run.smithi031.stdout:4736 trunc from 0xe34d29b to 0xe2cdf58 2024-09-13T07:13:20.947 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.947 INFO:teuthology.orchestra.run.smithi031.stdout:4738 read 0x639fa38 thru 0x63a8da1 (0x936a bytes) 2024-09-13T07:13:20.947 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.947 INFO:teuthology.orchestra.run.smithi031.stdout:4740 write 0x3163a9c thru 0x3163ff3 (0x558 bytes) 2024-09-13T07:13:20.948 INFO:teuthology.orchestra.run.smithi031.stdout:4742 write 0x15e7b7f thru 0x15ea99a (0x2e1c bytes) 2024-09-13T07:13:20.949 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:20.949+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:20.951 INFO:teuthology.orchestra.run.smithi031.stdout:4744 read 0x76ba1fa thru 0x76c1ca0 (0x7aa7 bytes) 2024-09-13T07:13:20.951 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.951 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.951 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.951 INFO:teuthology.orchestra.run.smithi031.stdout:4750 writesame 0xad309a5 thru 0xad34843 data_size 0x11(0x3e9f bytes) 2024-09-13T07:13:20.953 INFO:teuthology.orchestra.run.smithi031.stdout:4752 read 0x9c8554c thru 0x9c8d5f8 (0x80ad bytes) 2024-09-13T07:13:20.953 INFO:teuthology.orchestra.run.smithi031.stdout:4753 write 0x6039b53 thru 0x6047ff7 (0xe4a5 bytes) 2024-09-13T07:13:20.956 INFO:teuthology.orchestra.run.smithi031.stdout:4754 write 0x31f1bc1 thru 0x3201416 (0xf856 bytes) 2024-09-13T07:13:20.959 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.959 INFO:teuthology.orchestra.run.smithi031.stdout:4758 write 0x2e4db0b thru 0x2e57888 (0x9d7e bytes) 2024-09-13T07:13:20.963 INFO:teuthology.orchestra.run.smithi031.stdout:4761 writesame 0x371f517 thru 0x3721364 data_size 0x9(0x1e4e bytes) 2024-09-13T07:13:20.966 INFO:teuthology.orchestra.run.smithi031.stdout:4762 write 0x3636a18 thru 0x363f309 (0x88f2 bytes) 2024-09-13T07:13:20.981 INFO:teuthology.orchestra.run.smithi031.stdout:4765 trunc from 0xe2cdf58 to 0x3179722 2024-09-13T07:13:20.999 INFO:teuthology.orchestra.run.smithi031.stdout:4766 read 0x222a517 thru 0x22354af (0xaf99 bytes) 2024-09-13T07:13:20.999 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:20.999 INFO:teuthology.orchestra.run.smithi031.stdout:4768 read 0x20d1e7f thru 0x20d62d3 (0x4455 bytes) 2024-09-13T07:13:21.002 INFO:teuthology.orchestra.run.smithi031.stdout:4772 trunc from 0x3179722 to 0x5734cb 2024-09-13T07:13:21.014 INFO:teuthology.orchestra.run.smithi031.stdout:4773 read 0x3c26c5 thru 0x3cbe93 (0x97cf bytes) 2024-09-13T07:13:21.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.022 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.022 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.024 INFO:teuthology.orchestra.run.smithi031.stdout:4778 write 0x41328dc thru 0x4135291 (0x29b6 bytes) 2024-09-13T07:13:21.028 INFO:teuthology.orchestra.run.smithi031.stdout:4779 read 0x268038f thru 0x268df69 (0xdbdb bytes) 2024-09-13T07:13:21.028 INFO:teuthology.orchestra.run.smithi031.stdout:4780 read 0x12ca6c8 thru 0x12d8bc5 (0xe4fe bytes) 2024-09-13T07:13:21.028 INFO:teuthology.orchestra.run.smithi031.stdout:4781 write 0x21f0d70 thru 0x21f44a4 (0x3735 bytes) 2024-09-13T07:13:21.031 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:21.032 INFO:teuthology.orchestra.run.smithi031.stdout:4782 punch from 0x31cd874 to 0x31ce859, (0xfe5 bytes) 2024-09-13T07:13:21.032 INFO:teuthology.orchestra.run.smithi031.stdout:4786 punch from 0x2331382 to 0x233d65d, (0xc2db bytes) 2024-09-13T07:13:21.039 INFO:teuthology.orchestra.run.smithi031.stdout:4787 write 0x71ec36b thru 0x71f001c (0x3cb2 bytes) 2024-09-13T07:13:21.050 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:21.050+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:21.051 INFO:teuthology.orchestra.run.smithi031.stdout:4788 trunc from 0x71f001d to 0xe468979 2024-09-13T07:13:21.056 INFO:teuthology.orchestra.run.smithi031.stdout:4789 write 0x27425b8 thru 0x275022e (0xdc77 bytes) 2024-09-13T07:13:21.061 INFO:teuthology.orchestra.run.smithi031.stdout:4790 read 0x45f023f thru 0x45fab86 (0xa948 bytes) 2024-09-13T07:13:21.061 INFO:teuthology.orchestra.run.smithi031.stdout:4792 punch from 0x25857c7 to 0x259269e, (0xced7 bytes) 2024-09-13T07:13:21.061 INFO:teuthology.orchestra.run.smithi031.stdout:4793 writesame 0x38e1159 thru 0x38e4fee data_size 0x1(0x3e96 bytes) 2024-09-13T07:13:21.069 INFO:teuthology.orchestra.run.smithi031.stdout:4794 write 0xdf62cc4 thru 0xdf631e4 (0x521 bytes) 2024-09-13T07:13:21.072 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.072 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.072 INFO:teuthology.orchestra.run.smithi031.stdout:4798 read 0x3b390de thru 0x3b45959 (0xc87c bytes) 2024-09-13T07:13:21.073 INFO:teuthology.orchestra.run.smithi031.stdout:4799 write 0x15e8da0 thru 0x15f5e0e (0xd06f bytes) 2024-09-13T07:13:21.083 INFO:teuthology.orchestra.run.smithi031.stdout:4801 trunc from 0xe468979 to 0x771de94 2024-09-13T07:13:21.100 INFO:teuthology.orchestra.run.smithi031.stdout:4802 trunc from 0x771de94 to 0xb1ea3c 2024-09-13T07:13:21.111 INFO:teuthology.orchestra.run.smithi031.stdout:4803 read 0x3e9cdf thru 0x3e9edb (0x1fd bytes) 2024-09-13T07:13:21.132 INFO:teuthology.orchestra.run.smithi031.stdout:4804 write 0xdf22f56 thru 0xdf266bc (0x3767 bytes) 2024-09-13T07:13:21.135 INFO:teuthology.orchestra.run.smithi031.stdout:4806 write 0x3ca5ef0 thru 0x3caf6ad (0x97be bytes) 2024-09-13T07:13:21.138 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.138 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.143 INFO:teuthology.orchestra.run.smithi031.stdout:4807 write 0xe779ad7 thru 0xe77d3b5 (0x38df bytes) 2024-09-13T07:13:21.146 INFO:teuthology.orchestra.run.smithi031.stdout:4810 read 0xd72fa68 thru 0xd735fb4 (0x654d bytes) 2024-09-13T07:13:21.146 INFO:teuthology.orchestra.run.smithi031.stdout:4811 read 0x8c4b70f thru 0x8c56eed (0xb7df bytes) 2024-09-13T07:13:21.146 INFO:teuthology.orchestra.run.smithi031.stdout:4813 read 0xac6c65e thru 0xac6cf62 (0x905 bytes) 2024-09-13T07:13:21.148 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:21.150 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:21.151+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:21.159 INFO:teuthology.orchestra.run.smithi031.stdout:4814 trunc from 0xe77d3b6 to 0x327d8bc 2024-09-13T07:13:21.170 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.170 INFO:teuthology.orchestra.run.smithi031.stdout:4818 punch from 0x1399b0f to 0x139a6fb, (0xbec bytes) 2024-09-13T07:13:21.190 INFO:teuthology.orchestra.run.smithi031.stdout:4819 write 0xec1f231 thru 0xec2e21f (0xefef bytes) 2024-09-13T07:13:21.193 INFO:teuthology.orchestra.run.smithi031.stdout:4822 punch from 0xae92596 to 0xae9b596, (0x9000 bytes) 2024-09-13T07:13:21.193 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.193 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.194 INFO:teuthology.orchestra.run.smithi031.stdout:4825 read 0x2abdca5 thru 0x2abf164 (0x14c0 bytes) 2024-09-13T07:13:21.194 INFO:teuthology.orchestra.run.smithi031.stdout:4826 punch from 0x369cc34 to 0x36a5be8, (0x8fb4 bytes) 2024-09-13T07:13:21.194 INFO:teuthology.orchestra.run.smithi031.stdout:4827 punch from 0xc2796a1 to 0xc27c5d5, (0x2f34 bytes) 2024-09-13T07:13:21.194 INFO:teuthology.orchestra.run.smithi031.stdout:4828 read 0x435d3f7 thru 0x4365c17 (0x8821 bytes) 2024-09-13T07:13:21.194 INFO:teuthology.orchestra.run.smithi031.stdout:4829 write 0x715c32c thru 0x716a049 (0xdd1e bytes) 2024-09-13T07:13:21.196 INFO:teuthology.orchestra.run.smithi031.stdout:4830 writesame 0x75744ea thru 0x7578d49 data_size 0x30(0x4860 bytes) 2024-09-13T07:13:21.199 INFO:teuthology.orchestra.run.smithi031.stdout:4831 read 0xb3e81c6 thru 0xb3eea14 (0x684f bytes) 2024-09-13T07:13:21.199 INFO:teuthology.orchestra.run.smithi031.stdout:4832 write 0xbe869fc thru 0xbe909b6 (0x9fbb bytes) 2024-09-13T07:13:21.202 INFO:teuthology.orchestra.run.smithi031.stdout:4835 read 0x66b3839 thru 0x66c00b6 (0xc87e bytes) 2024-09-13T07:13:21.202 INFO:teuthology.orchestra.run.smithi031.stdout:4837 read 0x1c59c77 thru 0x1c67368 (0xd6f2 bytes) 2024-09-13T07:13:21.202 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.202 INFO:teuthology.orchestra.run.smithi031.stdout:4839 read 0xbfd1fea thru 0xbfddaaf (0xbac6 bytes) 2024-09-13T07:13:21.202 INFO:teuthology.orchestra.run.smithi031.stdout:4840 writesame 0x37bef0b thru 0x37c9ee3 data_size 0x3b(0xafd9 bytes) 2024-09-13T07:13:21.205 INFO:teuthology.orchestra.run.smithi031.stdout:4841 write 0x228c618 thru 0x2298450 (0xbe39 bytes) 2024-09-13T07:13:21.208 INFO:teuthology.orchestra.run.smithi031.stdout:4842 write 0xec135b1 thru 0xec1634f (0x2d9f bytes) 2024-09-13T07:13:21.210 INFO:teuthology.orchestra.run.smithi031.stdout:4844 write 0x6c518b7 thru 0x6c54f8a (0x36d4 bytes) 2024-09-13T07:13:21.214 INFO:teuthology.orchestra.run.smithi031.stdout:4845 read 0x2a65744 thru 0x2a7547d (0xfd3a bytes) 2024-09-13T07:13:21.214 INFO:teuthology.orchestra.run.smithi031.stdout:4847 write 0x6a95ffb thru 0x6a9badf (0x5ae5 bytes) 2024-09-13T07:13:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:4848 punch from 0xba57005 to 0xba5f042, (0x803d bytes) 2024-09-13T07:13:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:4849 read 0xc35be83 thru 0xc36761a (0xb798 bytes) 2024-09-13T07:13:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:4852 read 0xb53381c thru 0xb53ea96 (0xb27b bytes) 2024-09-13T07:13:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:4853 punch from 0xb389319 to 0xb391e68, (0x8b4f bytes) 2024-09-13T07:13:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:4854 write 0x242164c thru 0x242b318 (0x9ccd bytes) 2024-09-13T07:13:21.221 INFO:teuthology.orchestra.run.smithi031.stdout:4855 read 0x8c62718 thru 0x8c71e47 (0xf730 bytes) 2024-09-13T07:13:21.221 INFO:teuthology.orchestra.run.smithi031.stdout:4856 write 0x8cbf2de thru 0x8cc05ff (0x1322 bytes) 2024-09-13T07:13:21.224 INFO:teuthology.orchestra.run.smithi031.stdout:4857 write 0x104f5e3 thru 0x105bd87 (0xc7a5 bytes) 2024-09-13T07:13:21.227 INFO:teuthology.orchestra.run.smithi031.stdout:4858 read 0x229299d thru 0x22a17a8 (0xee0c bytes) 2024-09-13T07:13:21.228 INFO:teuthology.orchestra.run.smithi031.stdout:4860 write 0x17e0845 thru 0x17e7591 (0x6d4d bytes) 2024-09-13T07:13:21.231 INFO:teuthology.orchestra.run.smithi031.stdout:4863 read 0xb2707c4 thru 0xb27eee5 (0xe722 bytes) 2024-09-13T07:13:21.231 INFO:teuthology.orchestra.run.smithi031.stdout:4864 write 0xd2bb9d2 thru 0xd2c54cc (0x9afb bytes) 2024-09-13T07:13:21.235 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.235 INFO:teuthology.orchestra.run.smithi031.stdout:4867 punch from 0x7a3e169 to 0x7a40c1f, (0x2ab6 bytes) 2024-09-13T07:13:21.235 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.235 INFO:teuthology.orchestra.run.smithi031.stdout:4870 read 0x574d8de thru 0x575568b (0x7dae bytes) 2024-09-13T07:13:21.235 INFO:teuthology.orchestra.run.smithi031.stdout:4871 read 0x5427e9c thru 0x5432f3e (0xb0a3 bytes) 2024-09-13T07:13:21.235 INFO:teuthology.orchestra.run.smithi031.stdout:4872 write 0x8a5f78b thru 0x8a6752d (0x7da3 bytes) 2024-09-13T07:13:21.238 INFO:teuthology.orchestra.run.smithi031.stdout:4873 read 0x937909b thru 0x938550e (0xc474 bytes) 2024-09-13T07:13:21.238 INFO:teuthology.orchestra.run.smithi031.stdout:4874 punch from 0x17f7f87 to 0x17faa3b, (0x2ab4 bytes) 2024-09-13T07:13:21.238 INFO:teuthology.orchestra.run.smithi031.stdout:4875 punch from 0x9b935d9 to 0x9b9f26c, (0xbc93 bytes) 2024-09-13T07:13:21.238 INFO:teuthology.orchestra.run.smithi031.stdout:4876 write 0x31ffabb thru 0x320a4ad (0xa9f3 bytes) 2024-09-13T07:13:21.243 INFO:teuthology.orchestra.run.smithi031.stdout:4877 writesame 0x5df18af thru 0x5df83f0 data_size 0x1(0x6b42 bytes) 2024-09-13T07:13:21.251 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:21.252+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:21.258 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.258 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.259 INFO:teuthology.orchestra.run.smithi031.stdout:4879 punch from 0x9ba17c8 to 0x9bae141, (0xc979 bytes) 2024-09-13T07:13:21.259 INFO:teuthology.orchestra.run.smithi031.stdout:4880 read 0x72eb079 thru 0x72ec2eb (0x1273 bytes) 2024-09-13T07:13:21.259 INFO:teuthology.orchestra.run.smithi031.stdout:4881 read 0x9d1e2d7 thru 0x9d23600 (0x532a bytes) 2024-09-13T07:13:21.259 INFO:teuthology.orchestra.run.smithi031.stdout:4882 write 0xe4e4844 thru 0xe4f1afe (0xd2bb bytes) 2024-09-13T07:13:21.268 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:21.273 INFO:teuthology.orchestra.run.smithi031.stdout:4883 trunc from 0xec2e220 to 0x5029b37 2024-09-13T07:13:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:4884 punch from 0x3422fa3 to 0x342f14a, (0xc1a7 bytes) 2024-09-13T07:13:21.297 INFO:teuthology.orchestra.run.smithi031.stdout:4885 write 0xb5f95a6 thru 0xb602252 (0x8cad bytes) 2024-09-13T07:13:21.309 INFO:teuthology.orchestra.run.smithi031.stdout:4886 write 0xe22c991 thru 0xe234aab (0x811b bytes) 2024-09-13T07:13:21.312 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.312 INFO:teuthology.orchestra.run.smithi031.stdout:4888 read 0xb7bfd8c thru 0xb7c8979 (0x8bee bytes) 2024-09-13T07:13:21.313 INFO:teuthology.orchestra.run.smithi031.stdout:4890 read 0x185a234 thru 0x18627a3 (0x8570 bytes) 2024-09-13T07:13:21.313 INFO:teuthology.orchestra.run.smithi031.stdout:4892 write 0x926e4f thru 0x92c95a (0x5b0c bytes) 2024-09-13T07:13:21.316 INFO:teuthology.orchestra.run.smithi031.stdout:4893 read 0x548f6a3 thru 0x549395e (0x42bc bytes) 2024-09-13T07:13:21.316 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.316 INFO:teuthology.orchestra.run.smithi031.stdout:4897 punch from 0x2881c59 to 0x28873b4, (0x575b bytes) 2024-09-13T07:13:21.316 INFO:teuthology.orchestra.run.smithi031.stdout:4898 trunc from 0xe234aac to 0xe385969 2024-09-13T07:13:21.330 INFO:teuthology.orchestra.run.smithi031.stdout:4900 trunc from 0xe385969 to 0x6917665 2024-09-13T07:13:21.343 INFO:teuthology.orchestra.run.smithi031.stdout:4901 write 0x61f98aa thru 0x6202776 (0x8ecd bytes) 2024-09-13T07:13:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:4902 read 0x63d2134 thru 0x63d38fd (0x17ca bytes) 2024-09-13T07:13:21.347 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.352 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:21.353+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:21.354 INFO:teuthology.orchestra.run.smithi031.stdout:4904 write 0x8f2921a thru 0x8f2db4b (0x4932 bytes) 2024-09-13T07:13:21.358 INFO:teuthology.orchestra.run.smithi031.stdout:4905 punch from 0x2cad4b8 to 0x2cb3ba2, (0x66ea bytes) 2024-09-13T07:13:21.358 INFO:teuthology.orchestra.run.smithi031.stdout:4906 read 0x5b76183 thru 0x5b7b841 (0x56bf bytes) 2024-09-13T07:13:21.358 INFO:teuthology.orchestra.run.smithi031.stdout:4907 punch from 0x814a039 to 0x81577ce, (0xd795 bytes) 2024-09-13T07:13:21.358 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.369 INFO:teuthology.orchestra.run.smithi031.stdout:4912 write 0xee02015 thru 0xee06cb1 (0x4c9d bytes) 2024-09-13T07:13:21.372 INFO:teuthology.orchestra.run.smithi031.stdout:4913 clone 27 order 23 su 2097152 sc 5 2024-09-13T07:13:21.418 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.418 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.427 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:21.453 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:21.454+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.530 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.538 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:21.554 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:21.555+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:21.594 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T07:13:21.596 INFO:teuthology.orchestra.run.smithi031.stdout:4916 read 0x47a6dc6 thru 0x47aaf31 (0x416c bytes) 2024-09-13T07:13:21.597 INFO:teuthology.orchestra.run.smithi031.stdout:4917 write 0xbca9096 thru 0xbcb44e7 (0xb452 bytes) 2024-09-13T07:13:21.607 INFO:teuthology.orchestra.run.smithi031.stdout:4918 write 0x251891c thru 0x251c303 (0x39e8 bytes) 2024-09-13T07:13:21.611 INFO:teuthology.orchestra.run.smithi031.stdout:4919 read 0x9ebe8d7 thru 0x9ebee79 (0x5a3 bytes) 2024-09-13T07:13:21.622 INFO:teuthology.orchestra.run.smithi031.stdout:4922 trunc from 0xee06cb2 to 0x5244d82 2024-09-13T07:13:21.646 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.646 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.650 INFO:teuthology.orchestra.run.smithi031.stdout:4923 writesame 0xc181d50 thru 0xc190644 data_size 0x67(0xe8f5 bytes) 2024-09-13T07:13:21.650 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.650 INFO:teuthology.orchestra.run.smithi031.stdout:4926 read 0x7401ce0 thru 0x7404128 (0x2449 bytes) 2024-09-13T07:13:21.654 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:21.655+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:21.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:21.659 INFO:teuthology.orchestra.run.smithi031.stdout:4927 trunc from 0xc190645 to 0x3e33b62 2024-09-13T07:13:21.677 INFO:teuthology.orchestra.run.smithi031.stdout:4928 read 0x2483784 thru 0x24895d7 (0x5e54 bytes) 2024-09-13T07:13:21.677 INFO:teuthology.orchestra.run.smithi031.stdout:4930 write 0x3d93beb thru 0x3d9454f (0x965 bytes) 2024-09-13T07:13:21.678 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.678 INFO:teuthology.orchestra.run.smithi031.stdout:4933 read 0x349ea41 thru 0x34a9f95 (0xb555 bytes) 2024-09-13T07:13:21.679 INFO:teuthology.orchestra.run.smithi031.stdout:4934 read 0xc422d2 thru 0xc4acee (0x8a1d bytes) 2024-09-13T07:13:21.679 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.680 INFO:teuthology.orchestra.run.smithi031.stdout:4936 punch from 0x1a646c7 to 0x1a6b677, (0x6fb0 bytes) 2024-09-13T07:13:21.680 INFO:teuthology.orchestra.run.smithi031.stdout:4937 punch from 0xfab26a to 0xfb1c56, (0x69ec bytes) 2024-09-13T07:13:21.680 INFO:teuthology.orchestra.run.smithi031.stdout:4938 write 0x154f132 thru 0x1553f83 (0x4e52 bytes) 2024-09-13T07:13:21.696 INFO:teuthology.orchestra.run.smithi031.stdout:4939 trunc from 0x3e33b62 to 0xc9054b8 2024-09-13T07:13:21.702 INFO:teuthology.orchestra.run.smithi031.stdout:4940 punch from 0xa387afb to 0xa38fe54, (0x8359 bytes) 2024-09-13T07:13:21.702 INFO:teuthology.orchestra.run.smithi031.stdout:4943 write 0xaca908d thru 0xacaeb53 (0x5ac7 bytes) 2024-09-13T07:13:21.705 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.705 INFO:teuthology.orchestra.run.smithi031.stdout:4946 read 0xc60ec22 thru 0xc61202d (0x340c bytes) 2024-09-13T07:13:21.705 INFO:teuthology.orchestra.run.smithi031.stdout:4947 punch from 0x394e233 to 0x395c457, (0xe224 bytes) 2024-09-13T07:13:21.705 INFO:teuthology.orchestra.run.smithi031.stdout:4948 write 0xc5f5eb0 thru 0xc603845 (0xd996 bytes) 2024-09-13T07:13:21.709 INFO:teuthology.orchestra.run.smithi031.stdout:4950 punch from 0x43b9135 to 0x43c7088, (0xdf53 bytes) 2024-09-13T07:13:21.709 INFO:teuthology.orchestra.run.smithi031.stdout:4951 writesame 0x498175f thru 0x498eba5 data_size 0x1(0xd447 bytes) 2024-09-13T07:13:21.734 INFO:teuthology.orchestra.run.smithi031.stdout:4952 read 0x475c6d7 thru 0x476b8f4 (0xf21e bytes) 2024-09-13T07:13:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4953 trunc from 0xc9054b8 to 0x2854026 2024-09-13T07:13:21.755 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:21.756+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:21.765 INFO:teuthology.orchestra.run.smithi031.stdout:4954 writesame 0x21f849a thru 0x21faba1 data_size 0x4(0x2708 bytes) 2024-09-13T07:13:21.777 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.777 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.789 INFO:teuthology.orchestra.run.smithi031.stdout:4960 write 0xebb09a2 thru 0xebb5ad5 (0x5134 bytes) 2024-09-13T07:13:21.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:21.792 INFO:teuthology.orchestra.run.smithi031.stdout:4961 write 0x4ec5c85 thru 0x4ecb57d (0x58f9 bytes) 2024-09-13T07:13:21.796 INFO:teuthology.orchestra.run.smithi031.stdout:4962 writesame 0xa5e9768 thru 0xa5f8103 data_size 0x2(0xe99c bytes) 2024-09-13T07:13:21.814 INFO:teuthology.orchestra.run.smithi031.stdout:4965 write 0xec994e thru 0xed3a6e (0xa121 bytes) 2024-09-13T07:13:21.818 INFO:teuthology.orchestra.run.smithi031.stdout:4966 read 0xde55a99 thru 0xde56520 (0xa88 bytes) 2024-09-13T07:13:21.818 INFO:teuthology.orchestra.run.smithi031.stdout:4967 write 0x10b8b77 thru 0x10b8e6f (0x2f9 bytes) 2024-09-13T07:13:21.820 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:21.820 INFO:teuthology.orchestra.run.smithi031.stdout:4969 writesame 0x8e146d8 thru 0x8e1fbee data_size 0x65(0xb517 bytes) 2024-09-13T07:13:21.823 INFO:teuthology.orchestra.run.smithi031.stdout:4970 read 0xb3f021 thru 0xb4923b (0xa21b bytes) 2024-09-13T07:13:21.823 INFO:teuthology.orchestra.run.smithi031.stdout:4972 clone 28 order 22 su 524288 sc 12 2024-09-13T07:13:21.857 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:21.857+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:21.901 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:21.901 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:21.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:21.957 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:21.958+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:22.056 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:22.056 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:22.057 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:22.059+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:22.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:22.159 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:22.160+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:22.171 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:22.171 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:22.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:22.260 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:22.261+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:22.286 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:22.286 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:22.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:22.378 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:22.379+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:22.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:13:22.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:22.448 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:22.449 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:22.450 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone27 from 0xebb5ad6 (overlap 0x2854026) to 0x2043351 2024-09-13T07:13:22.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:22.462 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:22.463+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:22.540 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T07:13:22.563 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:22.564+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:22.610 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:22.627 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:22.628 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:22.629 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:22.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:22.645 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:22.645 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:22.662 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:22.663 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:22.664+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:22.679 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:22.680 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:22.697 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:22.697 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:22.714 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:22.756 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:22.757 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:22.764 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:22.765+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:22.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:22.814 INFO:teuthology.orchestra.run.smithi031.stdout:4975 trunc from 0xebb5ad6 to 0xe5a613a 2024-09-13T07:13:22.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:22.845 INFO:teuthology.orchestra.run.smithi031.stdout:4977 read 0xcf2569f thru 0xcf2a9ee (0x5350 bytes) 2024-09-13T07:13:22.845 INFO:teuthology.orchestra.run.smithi031.stdout:4979 read 0xe2de4be thru 0xe2ebe89 (0xd9cc bytes) 2024-09-13T07:13:22.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:22.845 INFO:teuthology.orchestra.run.smithi031.stdout:4984 write 0x6192cc5 thru 0x61953ac (0x26e8 bytes) 2024-09-13T07:13:22.850 INFO:teuthology.orchestra.run.smithi031.stdout:4985 write 0x9c7aafc thru 0x9c85c5b (0xb160 bytes) 2024-09-13T07:13:22.854 INFO:teuthology.orchestra.run.smithi031.stdout:4987 read 0x87f7a4 thru 0x886777 (0x6fd4 bytes) 2024-09-13T07:13:22.855 INFO:teuthology.orchestra.run.smithi031.stdout:4990 read 0x1e036ec thru 0x1e07594 (0x3ea9 bytes) 2024-09-13T07:13:22.855 INFO:teuthology.orchestra.run.smithi031.stdout:4991 punch from 0x2ef36f9 to 0x2ef66e5, (0x2fec bytes) 2024-09-13T07:13:22.856 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:22.856 INFO:teuthology.orchestra.run.smithi031.stdout:4993 punch from 0x209d1af to 0x20a49da, (0x782b bytes) 2024-09-13T07:13:22.856 INFO:teuthology.orchestra.run.smithi031.stdout:4994 trunc from 0xe5a613a to 0xd6ed0ac 2024-09-13T07:13:22.865 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:22.865+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:22.866 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:22.866 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:22.867 INFO:teuthology.orchestra.run.smithi031.stdout:4998 writesame 0x741782f thru 0x74193e1 data_size 0x1(0x1bb3 bytes) 2024-09-13T07:13:22.872 INFO:teuthology.orchestra.run.smithi031.stdout:4999 write 0xbe74926 thru 0xbe81082 (0xc75d bytes) 2024-09-13T07:13:22.880 INFO:teuthology.orchestra.run.smithi031.stdout:5001 trunc from 0xd6ed0ac to 0x9d56088 2024-09-13T07:13:22.902 INFO:teuthology.orchestra.run.smithi031.stdout:5002 read 0x1060eb8 thru 0x1060f1d (0x66 bytes) 2024-09-13T07:13:22.902 INFO:teuthology.orchestra.run.smithi031.stdout:5003 punch from 0x219f782 to 0x21aa495, (0xad13 bytes) 2024-09-13T07:13:22.903 INFO:teuthology.orchestra.run.smithi031.stdout:5004 punch from 0x1a2d445 to 0x1a38780, (0xb33b bytes) 2024-09-13T07:13:22.903 INFO:teuthology.orchestra.run.smithi031.stdout:5006 punch from 0x8850f6c to 0x8859213, (0x82a7 bytes) 2024-09-13T07:13:22.906 INFO:teuthology.orchestra.run.smithi031.stdout:5007 trunc from 0x9d56088 to 0xd3d8c8d 2024-09-13T07:13:22.912 INFO:teuthology.orchestra.run.smithi031.stdout:5011 punch from 0x7732e30 to 0x7739226, (0x63f6 bytes) 2024-09-13T07:13:22.912 INFO:teuthology.orchestra.run.smithi031.stdout:5012 punch from 0x2b0b0ea to 0x2b17f15, (0xce2b bytes) 2024-09-13T07:13:22.912 INFO:teuthology.orchestra.run.smithi031.stdout:5013 write 0xd3a8eb6 thru 0xd3b05c5 (0x7710 bytes) 2024-09-13T07:13:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:5014 write 0xac731a0 thru 0xac8264d (0xf4ae bytes) 2024-09-13T07:13:22.920 INFO:teuthology.orchestra.run.smithi031.stdout:5015 trunc from 0xd3d8c8d to 0x9c3d9a3 2024-09-13T07:13:22.927 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.628802186999998, 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.627324162000001, 2024-09-13T07:13:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.624507113, 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.055078152999997, 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.042454556999999, 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:13:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.298872789000001, 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:13:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.298335561999998, 2024-09-13T07:13:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:13:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.159253925999998, 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4162 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'113, mlcod=29'110)", 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.112446208000001, 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028573499999999998, 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814278+0000", 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814277+0000", 2024-09-13T07:13:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814283+0000", 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.558e-06 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814293+0000", 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0299000000000001e-05 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814298+0000", 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814345+0000", 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7707999999999998e-05 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814365+0000", 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9836000000000001e-05 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817100+0000", 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027346969999999999 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.817135+0000", 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5194999999999998e-05 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.111906632999997, 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.084422527999997, 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.078408103999998, 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.026715809999999, 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:13:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.490491962, 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.730338783999997, 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:13:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:13:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.219930094999999, 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:64537 3.9 e47/19 3:97a74db9:::rbd_data.117332306d66.0000000000000002:head v 47'372, mlcod=45'370)", 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.307316663000002, 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092884539999999998, 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.624988+0000", 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055812279999999997 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625018+0000", 2024-09-13T07:13:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9496999999999999e-05 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625031+0000", 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625078+0000", 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7290000000000003e-05 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625096+0000", 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8091999999999999e-05 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628662+0000", 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035652449999999999 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628696+0000", 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4035999999999997e-05 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:71538 3.0 e50/19 3:014540de:::rbd_data.1173f6918137.0000000000000082:head v 50'331, mlcod=49'326)", 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.508168963999999, 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032256870000000001, 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418554+0000", 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418562+0000", 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4000000000000003e-06 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418573+0000", 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1685000000000001e-05 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418580+0000", 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9419999999999998e-06 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418611+0000", 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1053000000000002e-05 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418630+0000", 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421749+0000", 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031190620000000001 2024-09-13T07:13:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421781+0000", 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1905e-05 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:85070 3.0 e58/19 3:077f600c:::rbd_data.11732319a2dc.0000000000000001:head v 58'356, mlcod=57'354)", 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3109042369999999, 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029694840000000001, 2024-09-13T07:13:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615818+0000", 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616402+0000", 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058395300000000001 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616417+0000", 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4630000000000001e-05 2024-09-13T07:13:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616424+0000", 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616469+0000", 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4508000000000001e-05 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616499+0000", 2024-09-13T07:13:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0491999999999999e-05 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618737+0000", 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002238037 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618789+0000", 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2055999999999998e-05 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:86269 3.9 e58/19 3:96e8c531:::rbd_data.11732319a2dc.00000000000000bd:head v 58'427, mlcod=58'425)", 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.738740468, 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030219940000000001, 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187992+0000", 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1339999999999993e-06 2024-09-13T07:13:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188006+0000", 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939e-05 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188013+0000", 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188067+0000", 2024-09-13T07:13:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4190000000000001e-05 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188085+0000", 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309e-05 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.190969+0000", 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028840379999999998 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.191005+0000", 2024-09-13T07:13:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6319999999999998e-05 2024-09-13T07:13:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.016 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:23.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:23.018 INFO:teuthology.orchestra.run.smithi031.stdout:5016 write 0xc8eb4ff thru 0xc8ef702 (0x4204 bytes) 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:5017 trunc from 0xc8ef703 to 0x20eb8f5 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:5018 writesame 0x42b43de thru 0x42c34f6 data_size 0x1f(0xf119 bytes) 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:5019 read 0x294e72d thru 0x295687c (0x8150 bytes) 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:5020 trunc from 0x42c34f7 to 0xaa7faef 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:5022 read 0xb6b224 thru 0xb7326c (0x8049 bytes) 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:5023 write 0x9068303 thru 0x906c76e (0x446c bytes) 2024-09-13T07:13:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:5026 trunc from 0xaa7faef to 0x68385f2 2024-09-13T07:13:23.020 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:23.018+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:23.024 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:23.026 INFO:teuthology.orchestra.run.smithi031.stdout:5029 trunc from 0x68385f2 to 0x7cee4b3 2024-09-13T07:13:23.030 INFO:teuthology.orchestra.run.smithi031.stdout:5031 writesame 0x269e10a thru 0x26ac076 data_size 0x1(0xdf6d bytes) 2024-09-13T07:13:23.059 INFO:teuthology.orchestra.run.smithi031.stdout:5033 read 0x11634db thru 0x1169e76 (0x699c bytes) 2024-09-13T07:13:23.060 INFO:teuthology.orchestra.run.smithi031.stdout:5034 writesame 0x3a06eb2 thru 0x3a082dc data_size 0x1(0x142b bytes) 2024-09-13T07:13:23.065 INFO:teuthology.orchestra.run.smithi031.stdout:5036 read 0x538fb7f thru 0x539c1c0 (0xc642 bytes) 2024-09-13T07:13:23.065 INFO:teuthology.orchestra.run.smithi031.stdout:5037 trunc from 0x7cee4b3 to 0x7cf7df1 2024-09-13T07:13:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:5038 read 0x186322b thru 0x1869793 (0x6569 bytes) 2024-09-13T07:13:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:5039 write 0x7bf5e06 thru 0x7c0106b (0xb266 bytes) 2024-09-13T07:13:23.073 INFO:teuthology.orchestra.run.smithi031.stdout:5040 punch from 0x392e1c2 to 0x392e9fa, (0x838 bytes) 2024-09-13T07:13:23.083 INFO:teuthology.orchestra.run.smithi031.stdout:5042 trunc from 0x7cf7df1 to 0x73e48a 2024-09-13T07:13:23.094 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:23.095 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:23.095 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:13:23.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:13:23.100 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:23.100 INFO:teuthology.orchestra.run.smithi031.stdout:5045 clone 29 order 22 su 2097152 sc 6 2024-09-13T07:13:23.118 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:23.119+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:23.140 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:23.140 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:23.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:23.220 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:23.220+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:23.297 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:23.298 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:23.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:23.350 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:23.351+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:13:23.388 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 61, 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "f667c631-bb9d-47bf-97c6-7217d8893244", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T07:12:08.346514+0000", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T07:13:22.391179+0000", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T07:13:13.343075+0000", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T07:12:10.626165+0000", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T07:13:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-13T07:13:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:19.268005+0000", 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 17, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T07:13:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:20.513150+0000", 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T07:13:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T07:13:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T07:13:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T07:13:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:28.796583+0000", 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T07:13:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "61", 2024-09-13T07:13:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 31, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 61, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.5, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.5, 2024-09-13T07:13:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.5, 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.5, 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:13:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c29fb342-d45b-48c1-9694-dd01e2a01325", 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/1709347595", 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/1709347595", 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/1709347595", 2024-09-13T07:13:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/1709347595", 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4b979b5c-a78b-461d-b795-f6f5c25cf652", 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.855316162109375, 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 31, 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T07:13:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T07:13:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3300827638", 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3300827638", 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3300827638", 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3300827638", 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:13:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "11259f54-1134-495f-9281-d5132aff2780", 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T07:13:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3977233096", 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3977233096", 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3977233096", 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3977233096", 2024-09-13T07:13:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "a11be2be-3803-4a23-bbf8-40ea110ccf41", 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/3290545828", 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/3290545828", 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/3290545828", 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/3290545828", 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:13:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2b6941d9-29f5-4836-a61c-47ff27a7172e", 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6800", 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6801", 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6802", 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6803", 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6806", 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6807", 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6804", 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6805", 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.436 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6801/3543694628", 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6803/3543694628", 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6807/3543694628", 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6805/3543694628", 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:13:23.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "17e3a02d-28e3-4126-a8f2-c7783e524d62", 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 55, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6808", 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6810", 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6812", 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6814", 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6820", 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6821", 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.441 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6816", 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6818", 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6810/951246042", 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6814/951246042", 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6821/951246042", 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6818/951246042", 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ddf29f57-7733-42fa-94ed-015a529fb689", 2024-09-13T07:13:23.443 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6824", 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6825", 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6826", 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6827", 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.446 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6830", 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6831", 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.447 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6828", 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6829", 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.448 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6825/1941674146", 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6827/1941674146", 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6831/1941674146", 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6829/1941674146", 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d91e2eed-3bfb-4eb0-ac30-18770c106742", 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:23.449 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 13, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 30, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 53, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 31, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6809", 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6811", 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6813", 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6815", 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6822", 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6823", 2024-09-13T07:13:23.452 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6817", 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:23.453 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6819", 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6811/48890479", 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6815/48890479", 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6823/48890479", 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6819/48890479", 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.454 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.805494+0000", 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:13:23.455 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.817444+0000", 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.456 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.712083+0000", 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.739958+0000", 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.720767+0000", 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.458 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.715944+0000", 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.788853+0000", 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:23.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.809325+0000", 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 31 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T07:13:23.460 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T07:13:23.461 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.462 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:23.464 INFO:tasks.thrashosds.thrasher:j is {'epoch': 61, 'fsid': 'f667c631-bb9d-47bf-97c6-7217d8893244', 'created': '2024-09-13T07:12:08.346514+0000', 'modified': '2024-09-13T07:13:22.391179+0000', 'last_up_change': '2024-09-13T07:13:13.343075+0000', 'last_in_change': '2024-09-13T07:12:10.626165+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:12:19.268005+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:12:20.513150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:12:28.796583+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'c29fb342-d45b-48c1-9694-dd01e2a01325', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 1709347595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 1709347595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 1709347595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 1709347595}]}, 'public_addr': '172.21.15.31:6809/1709347595', 'cluster_addr': '172.21.15.31:6811/1709347595', 'heartbeat_back_addr': '172.21.15.31:6815/1709347595', 'heartbeat_front_addr': '172.21.15.31:6813/1709347595', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4b979b5c-a78b-461d-b795-f6f5c25cf652', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.855316162109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3300827638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3300827638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3300827638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3300827638}]}, 'public_addr': '172.21.15.31:6825/3300827638', 'cluster_addr': '172.21.15.31:6827/3300827638', 'heartbeat_back_addr': '172.21.15.31:6831/3300827638', 'heartbeat_front_addr': '172.21.15.31:6829/3300827638', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '11259f54-1134-495f-9281-d5132aff2780', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3977233096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3977233096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3977233096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3977233096}]}, 'public_addr': '172.21.15.31:6801/3977233096', 'cluster_addr': '172.21.15.31:6803/3977233096', 'heartbeat_back_addr': '172.21.15.31:6807/3977233096', 'heartbeat_front_addr': '172.21.15.31:6805/3977233096', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a11be2be-3803-4a23-bbf8-40ea110ccf41', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 3290545828}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 3290545828}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 3290545828}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 3290545828}]}, 'public_addr': '172.21.15.31:6817/3290545828', 'cluster_addr': '172.21.15.31:6819/3290545828', 'heartbeat_back_addr': '172.21.15.31:6823/3290545828', 'heartbeat_front_addr': '172.21.15.31:6821/3290545828', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2b6941d9-29f5-4836-a61c-47ff27a7172e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 3543694628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 3543694628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 3543694628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 3543694628}]}, 'public_addr': '172.21.15.184:6801/3543694628', 'cluster_addr': '172.21.15.184:6803/3543694628', 'heartbeat_back_addr': '172.21.15.184:6807/3543694628', 'heartbeat_front_addr': '172.21.15.184:6805/3543694628', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '17e3a02d-28e3-4126-a8f2-c7783e524d62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6810', 'nonce': 951246042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6814', 'nonce': 951246042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 951246042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6818', 'nonce': 951246042}]}, 'public_addr': '172.21.15.184:6810/951246042', 'cluster_addr': '172.21.15.184:6814/951246042', 'heartbeat_back_addr': '172.21.15.184:6821/951246042', 'heartbeat_front_addr': '172.21.15.184:6818/951246042', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ddf29f57-7733-42fa-94ed-015a529fb689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 1941674146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 1941674146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 1941674146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 1941674146}]}, 'public_addr': '172.21.15.184:6825/1941674146', 'cluster_addr': '172.21.15.184:6827/1941674146', 'heartbeat_back_addr': '172.21.15.184:6831/1941674146', 'heartbeat_front_addr': '172.21.15.184:6829/1941674146', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd91e2eed-3bfb-4eb0-ac30-18770c106742', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 30, 'up_from': 53, 'up_thru': 56, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6809', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 48890479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6813', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 48890479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 48890479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6817', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 48890479}]}, 'public_addr': '172.21.15.184:6811/48890479', 'cluster_addr': '172.21.15.184:6815/48890479', 'heartbeat_back_addr': '172.21.15.184:6823/48890479', 'heartbeat_front_addr': '172.21.15.184:6819/48890479', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.805494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.817444+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.712083+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.739958+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.720767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.715944+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.788853+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.809325+0000', 'dead_epoch': 31}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:13:23.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:13:23.465 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:13:23.466 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:23.466 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:23.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:23.466 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:23.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856298443, 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:13:23.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856273129000002, 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:23.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:13:23.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856238763, 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:13:23.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.479 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856209938999999, 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:13:23.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856191641999999, 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:13:23.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:13:23.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856166836, 2024-09-13T07:13:23.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:13:23.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856148962999999, 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:23.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856122622000001, 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:23.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.8560959, 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856078315000001, 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:13:23.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:13:23.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:13:23.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856051677, 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:13:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856034217000001, 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:13:23.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.519 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.856007258999998, 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:23.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:13:23.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.855981055000001, 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:13:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:13:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:23.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.855963063000001, 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:13:23.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:13:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.855871296, 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:13:23.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.855848995999999, 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:13:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.855817994999999, 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:13:23.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:13:23.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.855793239, 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:13:23.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.855775547, 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:13:23.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:13:23.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:23.554 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:23.556 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:23.557 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:23.464+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:23.558 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone28 2024-09-13T07:13:23.558 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T07:13:23.581 INFO:teuthology.orchestra.run.smithi031.stdout:5046 writesame 0x6312653 thru 0x6318862 data_size 0x18(0x6210 bytes) 2024-09-13T07:13:23.582 INFO:teuthology.orchestra.run.smithi031.stdout:5048 writesame 0x2da7a2c thru 0x2db0bc8 data_size 0x1(0x919d bytes) 2024-09-13T07:13:23.583 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:23.583+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:23.601 INFO:teuthology.orchestra.run.smithi031.stdout:5050 punch from 0x59ae57d to 0x59bafb7, (0xca3a bytes) 2024-09-13T07:13:23.601 INFO:teuthology.orchestra.run.smithi031.stdout:5051 write 0x493c4e7 thru 0x49476f4 (0xb20e bytes) 2024-09-13T07:13:23.605 INFO:teuthology.orchestra.run.smithi031.stdout:5052 write 0x1dea1ea thru 0x1df4900 (0xa717 bytes) 2024-09-13T07:13:23.608 INFO:teuthology.orchestra.run.smithi031.stdout:5053 writesame 0x155d980 thru 0x156c9db data_size 0x2(0xf05c bytes) 2024-09-13T07:13:23.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:23.625 INFO:teuthology.orchestra.run.smithi031.stdout:5057 punch from 0x864149 to 0x8662fb, (0x21b2 bytes) 2024-09-13T07:13:23.625 INFO:teuthology.orchestra.run.smithi031.stdout:5058 read 0x1079786 thru 0x1079f81 (0x7fc bytes) 2024-09-13T07:13:23.631 INFO:teuthology.orchestra.run.smithi031.stdout:5059 trunc from 0x6318863 to 0x417a2a 2024-09-13T07:13:23.655 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:23.665 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:23.666+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:23.669 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:23.669 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:23.670 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:23.670 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:23.670 INFO:teuthology.orchestra.run.smithi031.stdout:5064 write 0x95d8c92 thru 0x95e3965 (0xacd4 bytes) 2024-09-13T07:13:23.679 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:23.684 INFO:teuthology.orchestra.run.smithi031.stdout:5065 trunc from 0x95e3966 to 0x23e191 2024-09-13T07:13:23.698 INFO:teuthology.orchestra.run.smithi031.stdout:5066 read 0x21dcf8 thru 0x22a3d2 (0xc6db bytes) 2024-09-13T07:13:23.699 INFO:teuthology.orchestra.run.smithi031.stdout:5067 punch from 0x219d2e to 0x221880, (0x7b52 bytes) 2024-09-13T07:13:23.699 INFO:teuthology.orchestra.run.smithi031.stdout:5071 read 0xba972 thru 0xbaad7 (0x166 bytes) 2024-09-13T07:13:23.703 INFO:teuthology.orchestra.run.smithi031.stdout:5073 trunc from 0x23e191 to 0x304bb4a 2024-09-13T07:13:23.707 INFO:teuthology.orchestra.run.smithi031.stdout:5074 punch from 0x14039d1 to 0x1411dbf, (0xe3ee bytes) 2024-09-13T07:13:23.766 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:23.767+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:23.820 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:23.820 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:23.820 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:23.820 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:23.820 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:23.828 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:23.869 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:23.870+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:13:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:13:23.893 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-13T07:13:23.410402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":819416805,"num_objects":1110,"num_object_clones":334,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1110,"num_whiteouts":385,"num_read":113122,"num_read_kb":489785,"num_write":13713,"num_write_kb":280995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":30321983,"num_keys_recovered":48,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10111,"ondisk_log_size":10111,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356088,"kb_used_data":44796,"kb_used_omap":20,"kb_used_meta":311147,"kb_avail":754618632,"statfs":{"total":773094113280,"available":772729479168,"internally_reserved":0,"allocated":45871104,"data_stored":358137085,"data_compressed":10229405,"data_compressed_allocated":27467776,"data_compressed_original":344549809,"omap_allocated":21248,"internal_metadata":318614784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":152314369,"num_objects":193,"num_object_clones":86,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":94,"num_read":9076,"num_read_kb":109651,"num_write":1423,"num_write_kb":76110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":11631918,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.930947"},"pg_stats":[{"pgid":"3.1f","version":"58'181","reported_seq":2890,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.788280+0000","last_change":"2024-09-13T07:12:32.166874+0000","last_active":"2024-09-13T07:13:18.788280+0000","last_peered":"2024-09-13T07:13:18.788280+0000","last_clean":"2024-09-13T07:13:18.788280+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:13:18.788280+0000","last_undegraded":"2024-09-13T07:13:18.788280+0000","last_fullsized":"2024-09-13T07:13:18.788280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001055669,"stat_sum":{"num_bytes":20781703,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":2726,"num_read_kb":10866,"num_write":234,"num_write_kb":6421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"58'214","reported_seq":3291,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.524803+0000","last_change":"2024-09-13T07:12:32.154466+0000","last_active":"2024-09-13T07:13:18.524803+0000","last_peered":"2024-09-13T07:13:18.524803+0000","last_clean":"2024-09-13T07:13:18.524803+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:13:18.524803+0000","last_undegraded":"2024-09-13T07:13:18.524803+0000","last_fullsized":"2024-09-13T07:13:18.524803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058285399999999995,"stat_sum":{"num_bytes":27586141,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":3409,"num_read_kb":15894,"num_write":200,"num_write_kb":5401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"58'490","reported_seq":2095,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.339572+0000","last_change":"2024-09-13T07:12:32.162019+0000","last_active":"2024-09-13T07:13:19.339572+0000","last_peered":"2024-09-13T07:13:19.339572+0000","last_clean":"2024-09-13T07:13:19.339572+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:13:19.339572+0000","last_undegraded":"2024-09-13T07:13:19.339572+0000","last_fullsized":"2024-09-13T07:13:19.339572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093934900000000002,"stat_sum":{"num_bytes":28123595,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":3637,"num_read_kb":14055,"num_write":776,"num_write_kb":8369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"58'179","reported_seq":2402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.510302+0000","last_change":"2024-09-13T07:12:32.161811+0000","last_active":"2024-09-13T07:13:18.510302+0000","last_peered":"2024-09-13T07:13:18.510302+0000","last_clean":"2024-09-13T07:13:18.510302+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:13:18.510302+0000","last_undegraded":"2024-09-13T07:13:18.510302+0000","last_fullsized":"2024-09-13T07:13:18.510302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082332300000000005,"stat_sum":{"num_bytes":28626692,"num_objects":27,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2280,"num_read_kb":19771,"num_write":173,"num_write_kb":1977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"58'572","reported_seq":3878,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.493070+0000","last_change":"2024-09-13T07:12:32.166941+0000","last_active":"2024-09-13T07:13:19.493070+0000","last_peered":"2024-09-13T07:13:19.493070+0000","last_clean":"2024-09-13T07:13:19.493070+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:13:19.493070+0000","last_undegraded":"2024-09-13T07:13:19.493070+0000","last_fullsized":"2024-09-13T07:13:19.493070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001067222,"stat_sum":{"num_bytes":37703308,"num_objects":47,"num_object_clones":17,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":5915,"num_read_kb":32135,"num_write":781,"num_write_kb":6193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"58'198","reported_seq":2795,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.479009+0000","last_change":"2024-09-13T07:12:32.157122+0000","last_active":"2024-09-13T07:13:19.479009+0000","last_peered":"2024-09-13T07:13:19.479009+0000","last_clean":"2024-09-13T07:13:19.479009+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:13:19.479009+0000","last_undegraded":"2024-09-13T07:13:19.479009+0000","last_fullsized":"2024-09-13T07:13:19.479009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074061500000000002,"stat_sum":{"num_bytes":20394019,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":3081,"num_read_kb":17846,"num_write":214,"num_write_kb":5832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"58'250","reported_seq":4400,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.080280+0000","last_change":"2024-09-13T07:12:32.152866+0000","last_active":"2024-09-13T07:13:19.080280+0000","last_peered":"2024-09-13T07:13:19.080280+0000","last_clean":"2024-09-13T07:13:19.080280+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:13:19.080280+0000","last_undegraded":"2024-09-13T07:13:19.080280+0000","last_fullsized":"2024-09-13T07:13:19.080280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000412552,"stat_sum":{"num_bytes":25447708,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":4390,"num_read_kb":16411,"num_write":306,"num_write_kb":17945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"58'361","reported_seq":4376,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.883141+0000","last_change":"2024-09-13T07:12:32.154395+0000","last_active":"2024-09-13T07:13:18.883141+0000","last_peered":"2024-09-13T07:13:18.883141+0000","last_clean":"2024-09-13T07:13:18.883141+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:13:18.883141+0000","last_undegraded":"2024-09-13T07:13:18.883141+0000","last_fullsized":"2024-09-13T07:13:18.883141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054318499999999996,"stat_sum":{"num_bytes":23440957,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":6417,"num_read_kb":24386,"num_write":437,"num_write_kb":6628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"57'259","reported_seq":2362,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.129101+0000","last_change":"2024-09-13T07:12:32.167160+0000","last_active":"2024-09-13T07:13:18.129101+0000","last_peered":"2024-09-13T07:13:18.129101+0000","last_clean":"2024-09-13T07:13:18.129101+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:13:18.129101+0000","last_undegraded":"2024-09-13T07:13:18.129101+0000","last_fullsized":"2024-09-13T07:13:18.129101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013000990000000001,"stat_sum":{"num_bytes":16019121,"num_objects":39,"num_object_clones":13,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":12,"num_read":3174,"num_read_kb":14967,"num_write":249,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"58'169","reported_seq":1077,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.821815+0000","last_change":"2024-09-13T07:13:14.415606+0000","last_active":"2024-09-13T07:13:18.821815+0000","last_peered":"2024-09-13T07:13:18.821815+0000","last_clean":"2024-09-13T07:13:18.821815+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:13:18.821815+0000","last_undegraded":"2024-09-13T07:13:18.821815+0000","last_fullsized":"2024-09-13T07:13:18.821815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000795997,"stat_sum":{"num_bytes":23404989,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":818,"num_read_kb":7337,"num_write":148,"num_write_kb":3756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"60'228","reported_seq":4217,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T07:13:21.673756+0000","last_change":"2024-09-13T07:13:17.373195+0000","last_active":"2024-09-13T07:13:21.673756+0000","last_peered":"2024-09-13T07:13:21.673756+0000","last_clean":"2024-09-13T07:13:21.673756+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:13:21.673756+0000","last_undegraded":"2024-09-13T07:13:21.673756+0000","last_fullsized":"2024-09-13T07:13:21.673756+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027228899999999999,"stat_sum":{"num_bytes":21497076,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":4080,"num_read_kb":14560,"num_write":331,"num_write_kb":6958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"58'277","reported_seq":1683,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.634216+0000","last_change":"2024-09-13T07:13:17.374397+0000","last_active":"2024-09-13T07:13:18.634216+0000","last_peered":"2024-09-13T07:13:18.634216+0000","last_clean":"2024-09-13T07:13:18.634216+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:13:18.634216+0000","last_undegraded":"2024-09-13T07:13:18.634216+0000","last_fullsized":"2024-09-13T07:13:18.634216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035592099999999998,"stat_sum":{"num_bytes":40356097,"num_objects":37,"num_object_clones":13,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":2275,"num_read_kb":12566,"num_write":351,"num_write_kb":2728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":56,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:17.364098+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:13:17.364098+0000","last_peered":"2024-09-13T07:13:17.364098+0000","last_clean":"2024-09-13T07:13:17.364098+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:13:17.364098+0000","last_undegraded":"2024-09-13T07:13:17.364098+0000","last_fullsized":"2024-09-13T07:13:17.364098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"58'299","reported_seq":3353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.266468+0000","last_change":"2024-09-13T07:12:32.162053+0000","last_active":"2024-09-13T07:13:19.266468+0000","last_peered":"2024-09-13T07:13:19.266468+0000","last_clean":"2024-09-13T07:13:19.266468+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:13:19.266468+0000","last_undegraded":"2024-09-13T07:13:19.266468+0000","last_fullsized":"2024-09-13T07:13:19.266468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091572500000000005,"stat_sum":{"num_bytes":12791406,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":4597,"num_read_kb":12547,"num_write":378,"num_write_kb":1969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:17.364091+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:13:17.364091+0000","last_peered":"2024-09-13T07:13:17.364091+0000","last_clean":"2024-09-13T07:13:17.364091+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:13:17.364091+0000","last_undegraded":"2024-09-13T07:13:17.364091+0000","last_fullsized":"2024-09-13T07:13:17.364091+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"57'406","reported_seq":3424,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.124632+0000","last_change":"2024-09-13T07:12:32.153134+0000","last_active":"2024-09-13T07:13:18.124632+0000","last_peered":"2024-09-13T07:13:18.124632+0000","last_clean":"2024-09-13T07:13:18.124632+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:13:18.124632+0000","last_undegraded":"2024-09-13T07:13:18.124632+0000","last_fullsized":"2024-09-13T07:13:18.124632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047834,"stat_sum":{"num_bytes":18538228,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":4549,"num_read_kb":9890,"num_write":557,"num_write_kb":6594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:17.371727+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:13:17.371727+0000","last_peered":"2024-09-13T07:13:17.371727+0000","last_clean":"2024-09-13T07:13:17.371727+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:13:17.371727+0000","last_undegraded":"2024-09-13T07:13:17.371727+0000","last_fullsized":"2024-09-13T07:13:17.371727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"58'283","reported_seq":654,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.559357+0000","last_change":"2024-09-13T07:13:17.372505+0000","last_active":"2024-09-13T07:13:18.559357+0000","last_peered":"2024-09-13T07:13:18.559357+0000","last_clean":"2024-09-13T07:13:18.559357+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:13:18.559357+0000","last_undegraded":"2024-09-13T07:13:18.559357+0000","last_fullsized":"2024-09-13T07:13:18.559357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062209699999999995,"stat_sum":{"num_bytes":8295208,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1081,"num_read_kb":4323,"num_write":400,"num_write_kb":1737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:17.368779+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:13:17.368779+0000","last_peered":"2024-09-13T07:13:17.368779+0000","last_clean":"2024-09-13T07:13:17.368779+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:13:17.368779+0000","last_undegraded":"2024-09-13T07:13:17.368779+0000","last_fullsized":"2024-09-13T07:13:17.368779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"58'220","reported_seq":6769,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.865600+0000","last_change":"2024-09-13T07:12:32.166255+0000","last_active":"2024-09-13T07:13:18.865600+0000","last_peered":"2024-09-13T07:13:18.865600+0000","last_clean":"2024-09-13T07:13:18.865600+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:13:18.865600+0000","last_undegraded":"2024-09-13T07:13:18.865600+0000","last_fullsized":"2024-09-13T07:13:18.865600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062381999999999997,"stat_sum":{"num_bytes":15936233,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":6738,"num_read_kb":16244,"num_write":284,"num_write_kb":16647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'329","reported_seq":1986,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.313749+0000","last_change":"2024-09-13T07:12:32.166064+0000","last_active":"2024-09-13T07:13:19.313749+0000","last_peered":"2024-09-13T07:13:19.313749+0000","last_clean":"2024-09-13T07:13:19.313749+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:13:19.313749+0000","last_undegraded":"2024-09-13T07:13:19.313749+0000","last_fullsized":"2024-09-13T07:13:19.313749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044591299999999998,"stat_sum":{"num_bytes":13065804,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":2752,"num_read_kb":14692,"num_write":452,"num_write_kb":5854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T07:13:20.386183+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:13:20.386183+0000","last_peered":"2024-09-13T07:13:20.386183+0000","last_clean":"2024-09-13T07:13:20.386183+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:13:20.386183+0000","last_undegraded":"2024-09-13T07:13:20.386183+0000","last_fullsized":"2024-09-13T07:13:20.386183+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":58,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:17.447625+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:13:17.447625+0000","last_peered":"2024-09-13T07:13:17.447625+0000","last_clean":"2024-09-13T07:13:17.447625+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:13:17.447625+0000","last_undegraded":"2024-09-13T07:13:17.447625+0000","last_fullsized":"2024-09-13T07:13:17.447625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"58'357","reported_seq":1803,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:18.679878+0000","last_change":"2024-09-13T07:12:32.152818+0000","last_active":"2024-09-13T07:13:18.679878+0000","last_peered":"2024-09-13T07:13:18.679878+0000","last_clean":"2024-09-13T07:13:18.679878+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:13:18.679878+0000","last_undegraded":"2024-09-13T07:13:18.679878+0000","last_fullsized":"2024-09-13T07:13:18.679878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038429300000000002,"stat_sum":{"num_bytes":33919881,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":2486,"num_read_kb":15849,"num_write":542,"num_write_kb":6004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":38,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T07:13:20.386217+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:13:20.386217+0000","last_peered":"2024-09-13T07:13:20.386217+0000","last_clean":"2024-09-13T07:13:20.386217+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:13:20.386217+0000","last_undegraded":"2024-09-13T07:13:20.386217+0000","last_fullsized":"2024-09-13T07:13:20.386217+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"58'258","reported_seq":3323,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.330466+0000","last_change":"2024-09-13T07:12:32.156546+0000","last_active":"2024-09-13T07:13:19.330466+0000","last_peered":"2024-09-13T07:13:19.330466+0000","last_clean":"2024-09-13T07:13:19.330466+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:13:19.330466+0000","last_undegraded":"2024-09-13T07:13:19.330466+0000","last_fullsized":"2024-09-13T07:13:19.330466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000193666,"stat_sum":{"num_bytes":48205047,"num_objects":44,"num_object_clones":15,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":3544,"num_read_kb":17711,"num_write":402,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T07:13:17.372449+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:13:17.372449+0000","last_peered":"2024-09-13T07:13:17.372449+0000","last_clean":"2024-09-13T07:13:17.372449+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:13:17.372449+0000","last_undegraded":"2024-09-13T07:13:17.372449+0000","last_fullsized":"2024-09-13T07:13:17.372449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"58'315","reported_seq":4260,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.033399+0000","last_change":"2024-09-13T07:12:32.154699+0000","last_active":"2024-09-13T07:13:19.033399+0000","last_peered":"2024-09-13T07:13:19.033399+0000","last_clean":"2024-09-13T07:13:19.033399+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:13:19.033399+0000","last_undegraded":"2024-09-13T07:13:19.033399+0000","last_fullsized":"2024-09-13T07:13:19.033399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065879899999999995,"stat_sum":{"num_bytes":48170361,"num_objects":33,"num_object_clones":12,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":8,"num_read":5087,"num_read_kb":18022,"num_write":484,"num_write_kb":23931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":36,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T07:13:20.386089+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:13:20.386089+0000","last_peered":"2024-09-13T07:13:20.386089+0000","last_clean":"2024-09-13T07:13:20.386089+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:13:20.386089+0000","last_undegraded":"2024-09-13T07:13:20.386089+0000","last_fullsized":"2024-09-13T07:13:20.386089+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"58'307","reported_seq":1661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.427577+0000","last_change":"2024-09-13T07:12:32.154641+0000","last_active":"2024-09-13T07:13:19.427577+0000","last_peered":"2024-09-13T07:13:19.427577+0000","last_clean":"2024-09-13T07:13:19.427577+0000","last_became_active":"2024-09-13T07:12:30.146382+0000","last_became_peered":"2024-09-13T07:12:30.146382+0000","last_unstale":"2024-09-13T07:13:19.427577+0000","last_undegraded":"2024-09-13T07:13:19.427577+0000","last_fullsized":"2024-09-13T07:13:19.427577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:44.566231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074801200000000005,"stat_sum":{"num_bytes":19205419,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":9,"num_read":2537,"num_read_kb":11277,"num_write":381,"num_write_kb":7141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"58'420","reported_seq":677,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.221783+0000","last_change":"2024-09-13T07:12:32.156602+0000","last_active":"2024-09-13T07:13:19.221783+0000","last_peered":"2024-09-13T07:13:19.221783+0000","last_clean":"2024-09-13T07:13:19.221783+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:13:19.221783+0000","last_undegraded":"2024-09-13T07:13:19.221783+0000","last_fullsized":"2024-09-13T07:13:19.221783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030558400000000002,"stat_sum":{"num_bytes":11313955,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":1702,"num_read_kb":4822,"num_write":692,"num_write_kb":8910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"58'570","reported_seq":2883,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.096429+0000","last_change":"2024-09-13T07:12:32.193641+0000","last_active":"2024-09-13T07:13:19.096429+0000","last_peered":"2024-09-13T07:13:19.096429+0000","last_clean":"2024-09-13T07:13:19.096429+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:13:19.096429+0000","last_undegraded":"2024-09-13T07:13:19.096429+0000","last_fullsized":"2024-09-13T07:13:19.096429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023357299999999999,"stat_sum":{"num_bytes":18590519,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":5788,"num_read_kb":16301,"num_write":769,"num_write_kb":5151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"58'323","reported_seq":2219,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.499674+0000","last_change":"2024-09-13T07:12:32.154134+0000","last_active":"2024-09-13T07:13:19.499674+0000","last_peered":"2024-09-13T07:13:19.499674+0000","last_clean":"2024-09-13T07:13:19.499674+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:13:19.499674+0000","last_undegraded":"2024-09-13T07:13:19.499674+0000","last_fullsized":"2024-09-13T07:13:19.499674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438474,"stat_sum":{"num_bytes":32779436,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":3789,"num_read_kb":13894,"num_write":371,"num_write_kb":7641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"58'152","reported_seq":1760,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.327701+0000","last_change":"2024-09-13T07:12:32.161253+0000","last_active":"2024-09-13T07:13:19.327701+0000","last_peered":"2024-09-13T07:13:19.327701+0000","last_clean":"2024-09-13T07:13:19.327701+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:13:19.327701+0000","last_undegraded":"2024-09-13T07:13:19.327701+0000","last_fullsized":"2024-09-13T07:13:19.327701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000322453,"stat_sum":{"num_bytes":29953446,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":1658,"num_read_kb":15803,"num_write":247,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"60'225","reported_seq":1415,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T07:13:21.589957+0000","last_change":"2024-09-13T07:13:17.375265+0000","last_active":"2024-09-13T07:13:21.589957+0000","last_peered":"2024-09-13T07:13:21.589957+0000","last_clean":"2024-09-13T07:13:21.589957+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:13:21.589957+0000","last_undegraded":"2024-09-13T07:13:21.589957+0000","last_fullsized":"2024-09-13T07:13:21.589957+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034938500000000002,"stat_sum":{"num_bytes":11454273,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":1659,"num_read_kb":4023,"num_write":250,"num_write_kb":2986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"59'264","reported_seq":836,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T07:13:21.767950+0000","last_change":"2024-09-13T07:13:14.420801+0000","last_active":"2024-09-13T07:13:21.767950+0000","last_peered":"2024-09-13T07:13:21.767950+0000","last_clean":"2024-09-13T07:13:21.767950+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:13:21.767950+0000","last_undegraded":"2024-09-13T07:13:21.767950+0000","last_fullsized":"2024-09-13T07:13:21.767950+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050734399999999996,"stat_sum":{"num_bytes":23087415,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":7,"num_read":3483,"num_read_kb":20654,"num_write":325,"num_write_kb":7380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"59'875","reported_seq":6511,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T07:13:20.993072+0000","last_change":"2024-09-13T07:13:17.375323+0000","last_active":"2024-09-13T07:13:20.993072+0000","last_peered":"2024-09-13T07:13:20.993072+0000","last_clean":"2024-09-13T07:13:20.993072+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:13:20.993072+0000","last_undegraded":"2024-09-13T07:13:20.993072+0000","last_fullsized":"2024-09-13T07:13:20.993072+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052212899999999997,"stat_sum":{"num_bytes":31319890,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":9360,"num_read_kb":12962,"num_write":1545,"num_write_kb":23002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"58'230","reported_seq":1086,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.144783+0000","last_change":"2024-09-13T07:12:32.193735+0000","last_active":"2024-09-13T07:13:19.144783+0000","last_peered":"2024-09-13T07:13:19.144783+0000","last_clean":"2024-09-13T07:13:19.144783+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:13:19.144783+0000","last_undegraded":"2024-09-13T07:13:19.144783+0000","last_fullsized":"2024-09-13T07:13:19.144783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031914699999999999,"stat_sum":{"num_bytes":43041337,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":1281,"num_read_kb":17909,"num_write":295,"num_write_kb":20669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"60'229","reported_seq":3372,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T07:13:21.630506+0000","last_change":"2024-09-13T07:13:17.374529+0000","last_active":"2024-09-13T07:13:21.630506+0000","last_peered":"2024-09-13T07:13:21.630506+0000","last_clean":"2024-09-13T07:13:21.630506+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:13:21.630506+0000","last_undegraded":"2024-09-13T07:13:21.630506+0000","last_fullsized":"2024-09-13T07:13:21.630506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000333254,"stat_sum":{"num_bytes":31639545,"num_objects":42,"num_object_clones":15,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":3328,"num_read_kb":26875,"num_write":316,"num_write_kb":8717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"58'161","reported_seq":1530,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T07:13:19.482130+0000","last_change":"2024-09-13T07:13:17.374031+0000","last_active":"2024-09-13T07:13:19.482130+0000","last_peered":"2024-09-13T07:13:19.482130+0000","last_clean":"2024-09-13T07:13:19.482130+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:13:19.482130+0000","last_undegraded":"2024-09-13T07:13:19.482130+0000","last_fullsized":"2024-09-13T07:13:19.482130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043838299999999998,"stat_sum":{"num_bytes":21548148,"num_objects":33,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1250,"num_read_kb":12541,"num_write":168,"num_write_kb":4317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'390","reported_seq":3135,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T07:13:21.767905+0000","last_change":"2024-09-13T07:13:17.374820+0000","last_active":"2024-09-13T07:13:21.767905+0000","last_peered":"2024-09-13T07:13:21.767905+0000","last_clean":"2024-09-13T07:13:21.767905+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:13:21.767905+0000","last_undegraded":"2024-09-13T07:13:21.767905+0000","last_fullsized":"2024-09-13T07:13:21.767905+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393278,"stat_sum":{"num_bytes":32589461,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":4135,"num_read_kb":22552,"num_write":494,"num_write_kb":10729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":818826418,"num_objects":1106,"num_object_clones":334,"num_object_copies":2212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1106,"num_whiteouts":385,"num_read":113006,"num_read_kb":489685,"num_write":13552,"num_write_kb":278577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":30321983,"num_keys_recovered":48,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40677376,"data_stored":357825038,"data_compressed":10349380,"data_compressed_allocated":27717632,"data_compressed_original":346691025,"omap_allocated":7126,"internal_metadata":0},"log_size":9991,"ondisk_log_size":9991,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266691,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39256,"kb_used_data":6476,"kb_used_omap":9,"kb_used_meta":32758,"kb_avail":94332584,"statfs":{"total":96636764160,"available":96596566016,"internally_reserved":0,"allocated":6631424,"data_stored":50722041,"data_compressed":1312015,"data_compressed_allocated":4063232,"data_compressed_original":48851527,"omap_allocated":10093,"internal_metadata":33544339},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34224,"kb_used_data":3112,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337616,"statfs":{"total":96636764160,"available":96601718784,"internally_reserved":0,"allocated":3186688,"data_stored":25138299,"data_compressed":754655,"data_compressed_allocated":1974272,"data_compressed_original":24415678,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.23799999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.35399999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.38700000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.28699999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40260,"kb_used_data":5028,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331580,"statfs":{"total":96636764160,"available":96595537920,"internally_reserved":0,"allocated":5148672,"data_stored":33638344,"data_compressed":891663,"data_compressed_allocated":2756608,"data_compressed_original":31870117,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.499}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.441},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.32700000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.42699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76600,"kb_used_data":9392,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94295240,"statfs":{"total":96636764160,"available":96558325760,"internally_reserved":0,"allocated":9617408,"data_stored":82069969,"data_compressed":2146775,"data_compressed_allocated":6131712,"data_compressed_original":79381049,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.318},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.34899999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.24099999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.25600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45212,"kb_used_data":5896,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326628,"statfs":{"total":96636764160,"available":96590467072,"internally_reserved":0,"allocated":6037504,"data_stored":49881687,"data_compressed":1601126,"data_compressed_allocated":3457024,"data_compressed_original":47982036,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2265088,"data_stored":10601431,"data_compressed":195099,"data_compressed_allocated":970752,"data_compressed_original":9754002,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39552,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332288,"statfs":{"total":96636764160,"available":96596262912,"internally_reserved":0,"allocated":4440064,"data_stored":31459290,"data_compressed":1034749,"data_compressed_allocated":2580480,"data_compressed_original":30149265,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47652,"kb_used_data":8344,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94324188,"statfs":{"total":96636764160,"available":96587968512,"internally_reserved":0,"allocated":8544256,"data_stored":74626024,"data_compressed":2293323,"data_compressed_allocated":5533696,"data_compressed_original":72146135,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8032256,"data_stored":76738606,"data_compressed":2309580,"data_compressed_allocated":5689344,"data_compressed_original":74557742,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3706880,"data_stored":30566111,"data_compressed":1026229,"data_compressed_allocated":2543616,"data_compressed_original":29575825,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":11356533,"data_compressed":316545,"data_compressed_allocated":1150976,"data_compressed_original":10810770,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5341184,"data_stored":49485758,"data_compressed":1600851,"data_compressed_allocated":3452928,"data_compressed_original":47900061,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8929280,"data_stored":81754095,"data_compressed":2146775,"data_compressed_allocated":6131712,"data_compressed_original":79381049,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4513792,"data_stored":33323512,"data_compressed":891419,"data_compressed_allocated":2752512,"data_compressed_original":31816869,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2498560,"data_stored":24783411,"data_compressed":754444,"data_compressed_allocated":1970176,"data_compressed_original":24370622,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5873664,"data_stored":49817012,"data_compressed":1303537,"data_compressed_allocated":4026368,"data_compressed_original":48278087,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:13:23.896 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [0, 1] 2024-09-13T07:13:23.896 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.8', '0', '1'] 2024-09-13T07:13:23.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.8 0 1 2024-09-13T07:13:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:5075 write 0xd252360 thru 0xd2529f0 (0x691 bytes) 2024-09-13T07:13:23.938 INFO:teuthology.orchestra.run.smithi031.stdout:5076 trunc from 0xd2529f1 to 0x571e041 2024-09-13T07:13:23.950 INFO:teuthology.orchestra.run.smithi031.stdout:5077 punch from 0x1ca0f47 to 0x1ca22fb, (0x13b4 bytes) 2024-09-13T07:13:23.955 INFO:teuthology.orchestra.run.smithi031.stdout:5078 trunc from 0x571e041 to 0x1a3ca58 2024-09-13T07:13:23.968 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:23.969+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:23.975 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.759512802000003, 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:23.976 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:13:23.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:13:23.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:13:23.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52442 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3985408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385696867, 2024-09-13T07:13:23.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040126756, 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52442 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590093+0000", 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0799999999999996e-06 2024-09-13T07:13:23.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590096+0000", 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2679999999999999e-06 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590099+0000", 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.481e-06 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630022+0000", 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039923472000000002 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630106+0000", 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4264999999999994e-05 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630215+0000", 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108399 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52443 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385683956000001, 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040282897999999998, 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52443 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590102+0000", 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2130000000000001e-06 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590105+0000", 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.881e-06 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590107+0000", 2024-09-13T07:13:23.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6510000000000001e-06 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630253+0000", 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040145713999999999 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630327+0000", 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3813000000000006e-05 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630384+0000", 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6842999999999998e-05 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52444 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4034560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385674755, 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040488365999999998, 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:23.991 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52444 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590111+0000", 2024-09-13T07:13:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.176e-06 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590114+0000", 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7889999999999999e-06 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590116+0000", 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.435e-06 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630413+0000", 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040296998000000001 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630493+0000", 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9610999999999994e-05 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630598+0000", 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000105566 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52445 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4059136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385666662999999, 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040607740000000003, 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52445 2024-09-13T07:13:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:23.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590119+0000", 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.288e-06 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590122+0000", 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7030000000000002e-06 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590124+0000", 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4250000000000001e-06 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630628+0000", 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040503577999999998 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630671+0000", 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3458999999999998e-05 2024-09-13T07:13:23.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630726+0000", 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4490000000000002e-05 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52446 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4083712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385658518, 2024-09-13T07:13:24.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040820130000000003, 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52446 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590127+0000", 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.271e-06 2024-09-13T07:13:24.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590130+0000", 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7389999999999999e-06 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590132+0000", 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630757+0000", 2024-09-13T07:13:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040624614000000003 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630836+0000", 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8913999999999997e-05 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630946+0000", 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110398 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52447 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4108288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385650247000001, 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041064639, 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52447 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590134+0000", 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590136+0000", 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.266e-06 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590138+0000", 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.869e-06 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590141+0000", 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6699999999999998e-06 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630984+0000", 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040843276999999997 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631072+0000", 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7386000000000007e-05 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631199+0000", 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012740000000000001 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52448 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385641585999998, 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041275991999999997, 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.009 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52448 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590144+0000", 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2109999999999999e-06 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590147+0000", 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8820000000000001e-06 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590150+0000", 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631231+0000", 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041081307999999997 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631298+0000", 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7340000000000002e-05 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631419+0000", 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000120986 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52449 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4157440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385632927, 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041487405999999998, 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52449 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590151+0000", 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590153+0000", 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590156+0000", 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.807e-06 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590158+0000", 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.458e-06 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631453+0000", 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041294850000000001 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631534+0000", 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0908999999999994e-05 2024-09-13T07:13:24.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631639+0000", 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000105099 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52450 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4182016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.385607228000001, 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041663446999999999, 2024-09-13T07:13:24.018 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52450 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590177+0000", 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590179+0000", 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.705e-06 2024-09-13T07:13:24.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590182+0000", 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3689999999999998e-06 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590185+0000", 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631666+0000", 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041480994 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631741+0000", 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4666999999999995e-05 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631841+0000", 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100449 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53407 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3158016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.283652204999999, 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040141932999999998, 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53407 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692132+0000", 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692134+0000", 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692140+0000", 2024-09-13T07:13:24.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8080000000000001e-06 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692145+0000", 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732076+0000", 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039931543 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732162+0000", 2024-09-13T07:13:24.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5657999999999999e-05 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732275+0000", 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011256600000000001 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53408 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3182592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.283636154, 2024-09-13T07:13:24.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040246709999999998, 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53408 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692148+0000", 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692150+0000", 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.877e-06 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692156+0000", 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.015e-06 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692161+0000", 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.544e-06 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732301+0000", 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040140051000000003 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732342+0000", 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1325999999999997e-05 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732395+0000", 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3229999999999997e-05 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.031 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53409 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3207168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.283610947, 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040446375, 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.032 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53409 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692173+0000", 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692180+0000", 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2620000000000004e-06 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692186+0000", 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.861e-06 2024-09-13T07:13:24.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692191+0000", 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732426+0000", 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040234501999999998 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732513+0000", 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6991999999999997e-05 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732620+0000", 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107413 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53410 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3231744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.036 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.283589405000001, 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040545431999999999, 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53410 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692197+0000", 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1280000000000002e-06 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692203+0000", 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8080000000000001e-06 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692208+0000", 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2399999999999998e-06 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732647+0000", 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040438395000000002 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732686+0000", 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9573000000000002e-05 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732741+0000", 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4731000000000002e-05 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53421 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.281732014999999, 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040071946999999997, 2024-09-13T07:13:24.041 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53421 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.042 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694052+0000", 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694058+0000", 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7679999999999997e-06 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694064+0000", 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2700000000000001e-06 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694069+0000", 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734115+0000", 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040045348000000001 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734125+0000", 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0151e-05 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53425 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.281647833000001, 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040069779, 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53425 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:24.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694139+0000", 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1330000000000002e-06 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694145+0000", 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694149+0000", 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5680000000000001e-06 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734197+0000", 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040048148999999998 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734207+0000", 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3670000000000003e-06 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.050 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53427 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.281615125999998, 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040080377, 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53427 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694169+0000", 2024-09-13T07:13:24.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694172+0000", 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2359999999999999e-06 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694177+0000", 2024-09-13T07:13:24.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8150000000000002e-06 2024-09-13T07:13:24.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694182+0000", 2024-09-13T07:13:24.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6380000000000003e-06 2024-09-13T07:13:24.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734242+0000", 2024-09-13T07:13:24.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040059715000000003 2024-09-13T07:13:24.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734250+0000", 2024-09-13T07:13:24.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3429999999999996e-06 2024-09-13T07:13:24.056 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.057 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.057 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.057 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:71808 3.a e50/19 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head v 50'527, mlcod=50'525)", 2024-09-13T07:13:24.057 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:24.057 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.391406591999999, 2024-09-13T07:13:24.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041667003000000001, 2024-09-13T07:13:24.058 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.058 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:24.058 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584377+0000", 2024-09-13T07:13:24.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:24.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584385+0000", 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2949999999999994e-06 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584402+0000", 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6458999999999999e-05 2024-09-13T07:13:24.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584409+0000", 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584457+0000", 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8130000000000002e-05 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584475+0000", 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7677e-05 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626008+0000", 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041532931000000002 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626045+0000", 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7654999999999998e-05 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71817 3.f 3:f1787d65:::rbd_object_map.1173ab534c79:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.390514506000001, 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040930686000000001, 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71817 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.064 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585269+0000", 2024-09-13T07:13:24.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585275+0000", 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.237e-06 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585284+0000", 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0529999999999996e-06 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585288+0000", 2024-09-13T07:13:24.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5029999999999999e-06 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585318+0000", 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0045000000000001e-05 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585364+0000", 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5985000000000001e-05 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626068+0000", 2024-09-13T07:13:24.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040703678 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626114+0000", 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6508999999999998e-05 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626201+0000", 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6655000000000003e-05 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3:f56afaab:::rbd_object_map.117375db170a:head [read 0~112] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.6291895430000007, 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0257515829999999, 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346593+0000", 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:24.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346603+0000", 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0037999999999999e-05 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346626+0000", 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3169000000000001e-05 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346638+0000", 2024-09-13T07:13:24.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1712999999999999e-05 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346719+0000", 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0980000000000001e-05 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346730+0000", 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0657e-05 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348732+0000", 2024-09-13T07:13:24.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020022690000000001 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348743+0000", 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0431999999999999e-05 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.371984+0000", 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0232416790000001 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372159+0000", 2024-09-13T07:13:24.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017503500000000001 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372347+0000", 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018757299999999999 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.074 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:24.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:5079 write 0xc36899a thru 0xc3735ef (0xac56 bytes) 2024-09-13T07:13:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:5083 read 0x3b86a66 thru 0x3b8ed18 (0x82b3 bytes) 2024-09-13T07:13:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:5085 write 0x4d7c8e7 thru 0x4d8c2b8 (0xf9d2 bytes) 2024-09-13T07:13:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:5086 read 0x113cf01 thru 0x114a126 (0xd226 bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5087 punch from 0xa727cf to 0xa76f13, (0x4744 bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5089 writesame 0x4ddec5c thru 0x4ddfd7d data_size 0x6(0x1122 bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5092 punch from 0x574096b to 0x5747667, (0x6cfc bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5096 read 0x7bd0bc4 thru 0x7bd6b7c (0x5fb9 bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5098 flatten 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5101 write 0xd33912 thru 0xd42e53 (0xf542 bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5104 trunc from 0xc3735f0 to 0xe871774 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5106 write 0x64ef880 thru 0x64f5204 (0x5985 bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5107 write 0xb59e171 thru 0xb5a74ab (0x933b bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5108 punch from 0xd613770 to 0xd61c6f6, (0x8f86 bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5110 writesame 0x87cacd4 thru 0x87d86fd data_size 0x19(0xda2a bytes) 2024-09-13T07:13:24.080 INFO:teuthology.orchestra.run.smithi031.stdout:5114 read 0x20edf4d thru 0x20fb4a2 (0xd556 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5115 read 0x2c5e572 thru 0x2c661ac (0x7c3b bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5116 write 0xd460abe thru 0xd46d336 (0xc879 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5117 write 0xbbbed55 thru 0xbbc67fc (0x7aa8 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5121 write 0x6889238 thru 0x688cba0 (0x3969 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5122 write 0x6b5cccb thru 0x6b5f832 (0x2b68 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5123 write 0xc221b6 thru 0xc2de05 (0xbc50 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5124 read 0x13490e8 thru 0x1349f22 (0xe3b bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5125 write 0xcd68020 thru 0xcd6f168 (0x7149 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5126 writesame 0xc3c1da2 thru 0xc3ce811 data_size 0x9e(0xca70 bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5127 writesame 0xbad6bc4 thru 0xbae46a0 data_size 0x1(0xdadd bytes) 2024-09-13T07:13:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:5130 write 0x470e534 thru 0x471bd33 (0xd800 bytes) 2024-09-13T07:13:24.082 INFO:teuthology.orchestra.run.smithi031.stdout:5131 read 0xae8beb3 thru 0xae98628 (0xc776 bytes) 2024-09-13T07:13:24.082 INFO:teuthology.orchestra.run.smithi031.stdout:5132 trunc from 0xe871774 to 0x394806a 2024-09-13T07:13:24.082 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:24.076+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:24.095 INFO:teuthology.orchestra.run.smithi031.stdout:5134 read 0x3593bd8 thru 0x35a2bd6 (0xefff bytes) 2024-09-13T07:13:24.095 INFO:teuthology.orchestra.run.smithi031.stdout:5136 punch from 0x12b812 to 0x134d34, (0x9522 bytes) 2024-09-13T07:13:24.111 INFO:teuthology.orchestra.run.smithi031.stdout:5137 write 0xbf60916 thru 0xbf650ad (0x4798 bytes) 2024-09-13T07:13:24.114 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.114 INFO:teuthology.orchestra.run.smithi031.stdout:5141 read 0x978ee02 thru 0x978f4fe (0x6fd bytes) 2024-09-13T07:13:24.124 INFO:teuthology.orchestra.run.smithi031.stdout:5142 write 0xea7fa81 thru 0xea8510f (0x568f bytes) 2024-09-13T07:13:24.127 INFO:teuthology.orchestra.run.smithi031.stdout:5143 read 0x57ac7be thru 0x57ad6ce (0xf11 bytes) 2024-09-13T07:13:24.127 INFO:teuthology.orchestra.run.smithi031.stdout:5144 read 0xe8d47d8 thru 0xe8dc276 (0x7a9f bytes) 2024-09-13T07:13:24.132 INFO:teuthology.orchestra.run.smithi031.stdout:5145 trunc from 0xea85110 to 0xa047e70 2024-09-13T07:13:24.144 INFO:teuthology.orchestra.run.smithi031.stdout:5146 write 0x24aca58 thru 0x24b3614 (0x6bbd bytes) 2024-09-13T07:13:24.147 INFO:teuthology.orchestra.run.smithi031.stdout:5147 read 0x15aea7a thru 0x15b05e6 (0x1b6d bytes) 2024-09-13T07:13:24.148 INFO:teuthology.orchestra.run.smithi031.stdout:5148 write 0x1ec4819 thru 0x1ec5c23 (0x140b bytes) 2024-09-13T07:13:24.152 INFO:teuthology.orchestra.run.smithi031.stdout:5149 write 0x5ccaa8f thru 0x5cd9e3f (0xf3b1 bytes) 2024-09-13T07:13:24.155 INFO:teuthology.orchestra.run.smithi031.stdout:5150 read 0x977076f thru 0x97717e6 (0x1078 bytes) 2024-09-13T07:13:24.163 INFO:teuthology.orchestra.run.smithi031.stdout:5151 writesame 0xc8093ff thru 0xc80a784 data_size 0x31(0x1386 bytes) 2024-09-13T07:13:24.163 INFO:teuthology.orchestra.run.smithi031.stdout:5152 read 0xc4ad22a thru 0xc4bbf81 (0xed58 bytes) 2024-09-13T07:13:24.163 INFO:teuthology.orchestra.run.smithi031.stdout:5153 punch from 0x639e03c to 0x63ad289, (0xf24d bytes) 2024-09-13T07:13:24.163 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.163 INFO:teuthology.orchestra.run.smithi031.stdout:5155 punch from 0x62a79b to 0x634e8a, (0xa6ef bytes) 2024-09-13T07:13:24.164 INFO:teuthology.orchestra.run.smithi031.stdout:5156 read 0x8952455 thru 0x895475d (0x2309 bytes) 2024-09-13T07:13:24.164 INFO:teuthology.orchestra.run.smithi031.stdout:5159 writesame 0x263053d thru 0x263e70a data_size 0x7(0xe1ce bytes) 2024-09-13T07:13:24.169 INFO:teuthology.orchestra.run.smithi031.stdout:5160 read 0xbf23c1f thru 0xbf2df16 (0xa2f8 bytes) 2024-09-13T07:13:24.169 INFO:teuthology.orchestra.run.smithi031.stdout:5161 read 0x9201007 thru 0x920a532 (0x952c bytes) 2024-09-13T07:13:24.169 INFO:teuthology.orchestra.run.smithi031.stdout:5162 read 0x5f59e53 thru 0x5f5dfe0 (0x418e bytes) 2024-09-13T07:13:24.169 INFO:teuthology.orchestra.run.smithi031.stdout:5163 write 0xb7b5c60 thru 0xb7b6e35 (0x11d6 bytes) 2024-09-13T07:13:24.172 INFO:teuthology.orchestra.run.smithi031.stdout:5165 write 0xa7ddac thru 0xa814ff (0x3754 bytes) 2024-09-13T07:13:24.175 INFO:teuthology.orchestra.run.smithi031.stdout:5166 punch from 0x2821380 to 0x2822ecb, (0x1b4b bytes) 2024-09-13T07:13:24.175 INFO:teuthology.orchestra.run.smithi031.stdout:5167 write 0x8892a13 thru 0x889f1cc (0xc7ba bytes) 2024-09-13T07:13:24.176 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:24.177+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:24.178 INFO:teuthology.orchestra.run.smithi031.stdout:5168 write 0xbf5e391 thru 0xbf693a5 (0xb015 bytes) 2024-09-13T07:13:24.181 INFO:teuthology.orchestra.run.smithi031.stdout:5169 write 0xadd1eea thru 0xadd3166 (0x127d bytes) 2024-09-13T07:13:24.184 INFO:teuthology.orchestra.run.smithi031.stdout:5170 write 0x8a18560 thru 0x8a20b1a (0x85bb bytes) 2024-09-13T07:13:24.192 INFO:teuthology.orchestra.run.smithi031.stdout:5171 trunc from 0xc80a785 to 0x7596a33 2024-09-13T07:13:24.196 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:24.196 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:24.196 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:24.196 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:24.202 INFO:teuthology.orchestra.run.smithi031.stdout:5173 write 0x3d508f5 thru 0x3d564d0 (0x5bdc bytes) 2024-09-13T07:13:24.206 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:24.207 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.207 INFO:teuthology.orchestra.run.smithi031.stdout:5176 read 0x44fcab5 thru 0x4500864 (0x3db0 bytes) 2024-09-13T07:13:24.207 INFO:teuthology.orchestra.run.smithi031.stdout:5177 punch from 0x216c501 to 0x216c617, (0x116 bytes) 2024-09-13T07:13:24.208 INFO:teuthology.orchestra.run.smithi031.stdout:5178 read 0x100f72 thru 0x1108d5 (0xf964 bytes) 2024-09-13T07:13:24.208 INFO:teuthology.orchestra.run.smithi031.stdout:5179 trunc from 0x7596a33 to 0x657f783 2024-09-13T07:13:24.225 INFO:teuthology.orchestra.run.smithi031.stdout:5180 writesame 0xbf6a7ac thru 0xbf72d00 data_size 0x1d(0x8555 bytes) 2024-09-13T07:13:24.225 INFO:teuthology.orchestra.run.smithi031.stdout:5182 write 0x293d36c thru 0x2941fb9 (0x4c4e bytes) 2024-09-13T07:13:24.228 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.234 INFO:teuthology.orchestra.run.smithi031.stdout:5184 writesame 0xcd38e21 thru 0xcd40bd8 data_size 0x1(0x7db8 bytes) 2024-09-13T07:13:24.234 INFO:teuthology.orchestra.run.smithi031.stdout:5185 write 0x29fba57 thru 0x29fc61d (0xbc7 bytes) 2024-09-13T07:13:24.235 INFO:teuthology.orchestra.run.smithi031.stdout:5189 read 0xbc12b6b thru 0xbc1ef59 (0xc3ef bytes) 2024-09-13T07:13:24.235 INFO:teuthology.orchestra.run.smithi031.stdout:5190 punch from 0xb8b13be to 0xb8b24cb, (0x110d bytes) 2024-09-13T07:13:24.235 INFO:teuthology.orchestra.run.smithi031.stdout:5191 read 0x9ccdc50 thru 0x9cd5b26 (0x7ed7 bytes) 2024-09-13T07:13:24.245 INFO:teuthology.orchestra.run.smithi031.stdout:5192 trunc from 0xcd40bd9 to 0x4215128 2024-09-13T07:13:24.271 INFO:teuthology.orchestra.run.smithi031.stdout:5194 write 0xb348898 thru 0xb354d6e (0xc4d7 bytes) 2024-09-13T07:13:24.274 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.276 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:24.277+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:24.282 INFO:teuthology.orchestra.run.smithi031.stdout:5197 writesame 0xcfe7457 thru 0xcff40b4 data_size 0x65(0xcc5e bytes) 2024-09-13T07:13:24.295 INFO:teuthology.orchestra.run.smithi031.stdout:5198 trunc from 0xcff40b5 to 0x17d92d5 2024-09-13T07:13:24.308 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.308 INFO:teuthology.orchestra.run.smithi031.stdout:5200 read 0x12a0d36 thru 0x12ad3b5 (0xc680 bytes) 2024-09-13T07:13:24.327 INFO:teuthology.orchestra.run.smithi031.stdout:5201 write 0xba9ee7a thru 0xbaa6466 (0x75ed bytes) 2024-09-13T07:13:24.339 INFO:teuthology.orchestra.run.smithi031.stdout:5202 write 0xe6953c1 thru 0xe6a0560 (0xb1a0 bytes) 2024-09-13T07:13:24.341 INFO:teuthology.orchestra.run.smithi031.stdout:5203 punch from 0xe093b88 to 0xe09faff, (0xbf77 bytes) 2024-09-13T07:13:24.342 INFO:teuthology.orchestra.run.smithi031.stdout:5204 punch from 0x9aa0f40 to 0x9aa8d8a, (0x7e4a bytes) 2024-09-13T07:13:24.343 INFO:teuthology.orchestra.run.smithi031.stdout:5205 trunc from 0xe6a0561 to 0xcdb245f 2024-09-13T07:13:24.353 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.353 INFO:teuthology.orchestra.run.smithi031.stdout:5208 writesame 0x3e7797f thru 0x3e7b0f6 data_size 0x47(0x3778 bytes) 2024-09-13T07:13:24.357 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:24.357 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:24.357 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:24.357 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:24.357 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:24.367 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:24.369 INFO:teuthology.orchestra.run.smithi031.stdout:5209 trunc from 0xcdb245f to 0x1842bd2 2024-09-13T07:13:24.383 INFO:teuthology.orchestra.run.smithi031.stdout:5210 read 0x1a022e thru 0x1a95a8 (0x937b bytes) 2024-09-13T07:13:24.395 INFO:teuthology.orchestra.run.smithi031.stdout:5211 write 0x5420eb9 thru 0x542f29f (0xe3e7 bytes) 2024-09-13T07:13:24.411 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:24.412+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:24.416 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.8 pg_upmap mapping to [0,1] 2024-09-13T07:13:24.419 INFO:teuthology.orchestra.run.smithi031.stdout:5213 write 0xdbab88a thru 0xdbb83ff (0xcb76 bytes) 2024-09-13T07:13:24.425 INFO:teuthology.orchestra.run.smithi031.stdout:5214 punch from 0xc6fab5e to 0xc6fd972, (0x2e14 bytes) 2024-09-13T07:13:24.425 INFO:teuthology.orchestra.run.smithi031.stdout:5216 read 0x447799a thru 0x447c3b2 (0x4a19 bytes) 2024-09-13T07:13:24.425 INFO:teuthology.orchestra.run.smithi031.stdout:5219 punch from 0x51ff342 to 0x5200d0a, (0x19c8 bytes) 2024-09-13T07:13:24.425 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.435 INFO:teuthology.orchestra.run.smithi031.stdout:5221 trunc from 0xdbb8400 to 0x4712eb3 2024-09-13T07:13:24.455 INFO:teuthology.orchestra.run.smithi031.stdout:5222 trunc from 0x4712eb3 to 0x9803a28 2024-09-13T07:13:24.462 INFO:teuthology.orchestra.run.smithi031.stdout:5223 punch from 0x8204706 to 0x820a594, (0x5e8e bytes) 2024-09-13T07:13:24.462 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.462 INFO:teuthology.orchestra.run.smithi031.stdout:5225 read 0x4375272 thru 0x4384439 (0xf1c8 bytes) 2024-09-13T07:13:24.462 INFO:teuthology.orchestra.run.smithi031.stdout:5226 write 0x62a0ad8 thru 0x62aaf5b (0xa484 bytes) 2024-09-13T07:13:24.467 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.467 INFO:teuthology.orchestra.run.smithi031.stdout:5228 punch from 0x6937c83 to 0x693d9ef, (0x5d6c bytes) 2024-09-13T07:13:24.467 INFO:teuthology.orchestra.run.smithi031.stdout:5229 punch from 0x706a4a9 to 0x7076776, (0xc2cd bytes) 2024-09-13T07:13:24.467 INFO:teuthology.orchestra.run.smithi031.stdout:5230 read 0x7b55df0 thru 0x7b6338f (0xd5a0 bytes) 2024-09-13T07:13:24.467 INFO:teuthology.orchestra.run.smithi031.stdout:5231 writesame 0x5a93b92 thru 0x5a96493 data_size 0x1d(0x2902 bytes) 2024-09-13T07:13:24.477 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:24.477 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36784 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13295616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.814095322, 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028020702000000001, 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36784 2024-09-13T07:13:24.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663477+0000", 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663479+0000", 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1299999999999999e-06 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663485+0000", 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0430000000000004e-06 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663494+0000", 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.6220000000000004e-06 2024-09-13T07:13:24.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691423+0000", 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027928339 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691464+0000", 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1743999999999999e-05 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691498+0000", 2024-09-13T07:13:24.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3235999999999998e-05 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36785 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13320192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.814073745000002, 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028091378, 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.482 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36785 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663499+0000", 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663498+0000", 2024-09-13T07:13:24.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663500+0000", 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2400000000000002e-06 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663506+0000", 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663511+0000", 2024-09-13T07:13:24.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6689999999999997e-06 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691518+0000", 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028007641999999999 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691557+0000", 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8139000000000002e-05 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691590+0000", 2024-09-13T07:13:24.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3334000000000003e-05 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36786 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13344768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.814057355999999, 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028164842999999998, 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36786 2024-09-13T07:13:24.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663515+0000", 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663514+0000", 2024-09-13T07:13:24.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663521+0000", 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9970000000000001e-06 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663527+0000", 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8690000000000001e-06 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663532+0000", 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4229999999999998e-06 2024-09-13T07:13:24.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691609+0000", 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028077429000000001 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691647+0000", 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7540000000000003e-05 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691680+0000", 2024-09-13T07:13:24.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3161000000000003e-05 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36787 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13369344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.814028402000002, 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028232916, 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.490 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36787 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663544+0000", 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663546+0000", 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3070000000000001e-06 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663552+0000", 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1149999999999999e-06 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663556+0000", 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691702+0000", 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028145752999999999 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691741+0000", 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8611e-05 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691777+0000", 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5982999999999998e-05 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36788 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13393920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.814011863000001, 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028311981999999999, 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36788 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:24.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663560+0000", 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663565+0000", 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4519999999999996e-06 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663571+0000", 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5480000000000001e-06 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663576+0000", 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6079999999999998e-06 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691800+0000", 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028224468999999999 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691839+0000", 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8476999999999998e-05 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691872+0000", 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3868000000000002e-05 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36789 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13418496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.498 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813992729999999, 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028396060000000001, 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36789 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663580+0000", 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663579+0000", 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663581+0000", 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9369999999999998e-06 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663586+0000", 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2800000000000003e-06 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663590+0000", 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.031e-06 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691896+0000", 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028305815000000002 2024-09-13T07:13:24.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691941+0000", 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4956999999999997e-05 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691976+0000", 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4422999999999999e-05 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.502 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36790 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13443072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813977830999999, 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028477820000000001, 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36790 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:24.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663594+0000", 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663596+0000", 2024-09-13T07:13:24.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2390000000000001e-06 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663602+0000", 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6740000000000002e-06 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663607+0000", 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5159999999999996e-06 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.691998+0000", 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028391937999999999 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692036+0000", 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7948999999999999e-05 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692072+0000", 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5893999999999999e-05 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36791 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13467648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813961918, 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028558888000000001, 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36791 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.507 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663610+0000", 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663612+0000", 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9420000000000002e-06 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663617+0000", 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.49e-06 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663630+0000", 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2126e-05 2024-09-13T07:13:24.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692094+0000", 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028464382 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692134+0000", 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0234999999999997e-05 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692169+0000", 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5098999999999997e-05 2024-09-13T07:13:24.510 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36792 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13492224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813938581999999, 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028626243999999999, 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.511 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36792 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663634+0000", 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663633+0000", 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663636+0000", 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1909999999999999e-06 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663641+0000", 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7089999999999999e-06 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663646+0000", 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8339999999999998e-06 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692191+0000", 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028544491000000002 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692227+0000", 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6189999999999997e-05 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692260+0000", 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3198999999999998e-05 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36793 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13516800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813922199, 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028699295999999999, 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.515 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36793 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663650+0000", 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663656+0000", 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6220000000000003e-06 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663663+0000", 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1489999999999999e-06 2024-09-13T07:13:24.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663667+0000", 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5589999999999999e-06 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692280+0000", 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028612779000000001 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692316+0000", 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6492999999999998e-05 2024-09-13T07:13:24.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692349+0000", 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3052000000000001e-05 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813900295, 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.519 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:24.523 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813862644, 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.813843417000001, 2024-09-13T07:13:24.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36799 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13664256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.812643521999998, 2024-09-13T07:13:24.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027993130000000001, 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36799 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664929+0000", 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664928+0000", 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664930+0000", 2024-09-13T07:13:24.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0109999999999999e-06 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664936+0000", 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7810000000000002e-06 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664941+0000", 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5990000000000004e-06 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692841+0000", 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027900115999999999 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692887+0000", 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6063999999999998e-05 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692922+0000", 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4906000000000001e-05 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36800 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13688832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.812627562999999, 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028070522000000001, 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36800 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664945+0000", 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664944+0000", 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664950+0000", 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4949999999999999e-06 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664956+0000", 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6829999999999996e-06 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664967+0000", 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1858000000000001e-05 2024-09-13T07:13:24.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692944+0000", 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027976671000000002 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692981+0000", 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7311999999999998e-05 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693015+0000", 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3856999999999998e-05 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.539 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36801 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13713408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.81260052, 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028136613000000001, 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36801 2024-09-13T07:13:24.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664972+0000", 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.541 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664971+0000", 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664974+0000", 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4710000000000001e-06 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664980+0000", 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6620000000000002e-06 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.664984+0000", 2024-09-13T07:13:24.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.561e-06 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693038+0000", 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028054374999999999 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693075+0000", 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6740999999999999e-05 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693108+0000", 2024-09-13T07:13:24.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3207000000000002e-05 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36802 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13737984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.812569631999999, 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028199977000000001, 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.544 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36802 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665003+0000", 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.545 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665002+0000", 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665005+0000", 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5790000000000002e-06 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665011+0000", 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7089999999999996e-06 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665022+0000", 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0631e-05 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693129+0000", 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028106822 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693168+0000", 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9069999999999997e-05 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693203+0000", 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4826999999999997e-05 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36803 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13762560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.812545853, 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028271376000000001, 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.548 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36803 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665026+0000", 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665028+0000", 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.294e-06 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665034+0000", 2024-09-13T07:13:24.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5559999999999998e-06 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665038+0000", 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5179999999999998e-06 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693224+0000", 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028185387999999999 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693265+0000", 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0791999999999998e-05 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693298+0000", 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3198000000000003e-05 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36804 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13787136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:24.552 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.81252958, 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028345479, 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36804 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665043+0000", 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665042+0000", 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665044+0000", 2024-09-13T07:13:24.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.199e-06 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665050+0000", 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.733e-06 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.665054+0000", 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2119999999999997e-06 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693319+0000", 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028264105000000001 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693355+0000", 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6618000000000001e-05 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.693388+0000", 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3065999999999998e-05 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.937770005000001, 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:24.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:24.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:24.562 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:24.564 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:24.565 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:24.564+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:24.565 INFO:teuthology.orchestra.run.smithi031.stdout:5233 write 0x9f76c48 thru 0x9f85658 (0xea11 bytes) 2024-09-13T07:13:24.565 INFO:teuthology.orchestra.run.smithi031.stdout:5234 punch from 0x276c30a to 0x2772fb2, (0x6ca8 bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5235 write 0x85e3685 thru 0x85f0a2e (0xd3aa bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5237 read 0x815e3b6 thru 0x816521d (0x6e68 bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5239 trunc from 0x9f85659 to 0xe7dc32a 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5245 writesame 0x11084c4 thru 0x1113c13 data_size 0x38(0xb750 bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5246 read 0x19decef thru 0x19eeb21 (0xfe33 bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5247 write 0x342f5e8 thru 0x343abc2 (0xb5db bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5248 trunc from 0xe7dc32a to 0x293655b 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5251 writesame 0x2155518 thru 0x215fa6d data_size 0x1(0xa556 bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5252 write 0x7bde41 thru 0x7c3ee8 (0x60a8 bytes) 2024-09-13T07:13:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:5253 read 0xce1697 thru 0xcea3b7 (0x8d21 bytes) 2024-09-13T07:13:24.569 INFO:teuthology.orchestra.run.smithi031.stdout:5255 writesame 0xb64ef5b thru 0xb6560f1 data_size 0x1b(0x7197 bytes) 2024-09-13T07:13:24.569 INFO:teuthology.orchestra.run.smithi031.stdout:5256 write 0xbd2697 thru 0xbde99b (0xc305 bytes) 2024-09-13T07:13:24.579 INFO:teuthology.orchestra.run.smithi031.stdout:5257 write 0xe3baed1 thru 0xe3bc639 (0x1769 bytes) 2024-09-13T07:13:24.582 INFO:teuthology.orchestra.run.smithi031.stdout:5259 writesame 0x8aa0beb thru 0x8aa46b9 data_size 0x1(0x3acf bytes) 2024-09-13T07:13:24.592 INFO:teuthology.orchestra.run.smithi031.stdout:5260 write 0x3f90286 thru 0x3f93780 (0x34fb bytes) 2024-09-13T07:13:24.595 INFO:teuthology.orchestra.run.smithi031.stdout:5262 writesame 0x96c6ae1 thru 0x96c8fdd data_size 0x1(0x24fd bytes) 2024-09-13T07:13:24.601 INFO:teuthology.orchestra.run.smithi031.stdout:5263 write 0xb540922 thru 0xb540a28 (0x107 bytes) 2024-09-13T07:13:24.618 INFO:teuthology.orchestra.run.smithi031.stdout:5264 trunc from 0xe3bc63a to 0x15166c5 2024-09-13T07:13:24.636 INFO:teuthology.orchestra.run.smithi031.stdout:5265 trunc from 0x15166c5 to 0x66b5f13 2024-09-13T07:13:24.647 INFO:teuthology.orchestra.run.smithi031.stdout:5266 trunc from 0x66b5f13 to 0x2a70cab 2024-09-13T07:13:24.654 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.664 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:24.665+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:24.667 INFO:teuthology.orchestra.run.smithi031.stdout:5269 write 0x732e279 thru 0x733a264 (0xbfec bytes) 2024-09-13T07:13:24.670 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.681 INFO:teuthology.orchestra.run.smithi031.stdout:5272 write 0xbc25f25 thru 0xbc3275b (0xc837 bytes) 2024-09-13T07:13:24.685 INFO:teuthology.orchestra.run.smithi031.stdout:5275 write 0x94cb36b thru 0x94d19ee (0x6684 bytes) 2024-09-13T07:13:24.689 INFO:teuthology.orchestra.run.smithi031.stdout:5276 write 0x42101ed thru 0x421078b (0x59f bytes) 2024-09-13T07:13:24.692 INFO:teuthology.orchestra.run.smithi031.stdout:5277 read 0x80f8354 thru 0x8102684 (0xa331 bytes) 2024-09-13T07:13:24.701 INFO:teuthology.orchestra.run.smithi031.stdout:5278 writesame 0xc9f71ff thru 0xc9f8404 data_size 0x3(0x1206 bytes) 2024-09-13T07:13:24.701 INFO:teuthology.orchestra.run.smithi031.stdout:5281 read 0x8955124 thru 0x8956c97 (0x1b74 bytes) 2024-09-13T07:13:24.701 INFO:teuthology.orchestra.run.smithi031.stdout:5282 write 0x1523573 thru 0x152fcef (0xc77d bytes) 2024-09-13T07:13:24.707 INFO:teuthology.orchestra.run.smithi031.stdout:5283 trunc from 0xc9f8405 to 0x9a9c3f4 2024-09-13T07:13:24.717 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:24.717 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:24.718 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:24.718 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.719 INFO:teuthology.orchestra.run.smithi031.stdout:5289 writesame 0x43dc29f thru 0x43df049 data_size 0x9(0x2dab bytes) 2024-09-13T07:13:24.720 INFO:teuthology.orchestra.run.smithi031.stdout:5290 read 0x57690a0 thru 0x5775b67 (0xcac8 bytes) 2024-09-13T07:13:24.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:24.728 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:24.733 INFO:teuthology.orchestra.run.smithi031.stdout:5291 write 0xee6616c thru 0xee68fb1 (0x2e46 bytes) 2024-09-13T07:13:24.737 INFO:teuthology.orchestra.run.smithi031.stdout:5293 read 0xa2a5197 thru 0xa2af465 (0xa2cf bytes) 2024-09-13T07:13:24.737 INFO:teuthology.orchestra.run.smithi031.stdout:5294 read 0x27ad4ba thru 0x27bc8db (0xf422 bytes) 2024-09-13T07:13:24.737 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.737 INFO:teuthology.orchestra.run.smithi031.stdout:5296 writesame 0xe3d1f9f thru 0xe3dbb59 data_size 0x3(0x9bbb bytes) 2024-09-13T07:13:24.744 INFO:teuthology.orchestra.run.smithi031.stdout:5297 write 0x2ec5c78 thru 0x2ed453e (0xe8c7 bytes) 2024-09-13T07:13:24.747 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.760 INFO:teuthology.orchestra.run.smithi031.stdout:5300 trunc from 0xee68fb2 to 0x3972eee 2024-09-13T07:13:24.772 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:24.773+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:24.787 INFO:teuthology.orchestra.run.smithi031.stdout:5302 writesame 0x875d496 thru 0x8760685 data_size 0x2f(0x31f0 bytes) 2024-09-13T07:13:24.787 INFO:teuthology.orchestra.run.smithi031.stdout:5303 read 0x5c8cc88 thru 0x5c8e49b (0x1814 bytes) 2024-09-13T07:13:24.788 INFO:teuthology.orchestra.run.smithi031.stdout:5304 write 0x519ac33 thru 0x51a531f (0xa6ed bytes) 2024-09-13T07:13:24.791 INFO:teuthology.orchestra.run.smithi031.stdout:5305 write 0x273072d thru 0x2739e42 (0x9716 bytes) 2024-09-13T07:13:24.796 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.796 INFO:teuthology.orchestra.run.smithi031.stdout:5310 punch from 0x252e820 to 0x2531b36, (0x3316 bytes) 2024-09-13T07:13:24.807 INFO:teuthology.orchestra.run.smithi031.stdout:5312 write 0xc7f81e5 thru 0xc804cc5 (0xcae1 bytes) 2024-09-13T07:13:24.813 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.813 INFO:teuthology.orchestra.run.smithi031.stdout:5315 write 0x7d77554 thru 0x7d82a97 (0xb544 bytes) 2024-09-13T07:13:24.817 INFO:teuthology.orchestra.run.smithi031.stdout:5317 write 0x2d4de4f thru 0x2d53586 (0x5738 bytes) 2024-09-13T07:13:24.820 INFO:teuthology.orchestra.run.smithi031.stdout:5319 read 0x3c27628 thru 0x3c2f519 (0x7ef2 bytes) 2024-09-13T07:13:24.820 INFO:teuthology.orchestra.run.smithi031.stdout:5320 write 0xb98a30f thru 0xb99336a (0x905c bytes) 2024-09-13T07:13:24.823 INFO:teuthology.orchestra.run.smithi031.stdout:5321 writesame 0x5564c00 thru 0x556b645 data_size 0x2(0x6a46 bytes) 2024-09-13T07:13:24.832 INFO:teuthology.orchestra.run.smithi031.stdout:5322 punch from 0x435c1ff to 0x43668a0, (0xa6a1 bytes) 2024-09-13T07:13:24.832 INFO:teuthology.orchestra.run.smithi031.stdout:5324 read 0xb48fc9 thru 0xb4adb0 (0x1de8 bytes) 2024-09-13T07:13:24.835 INFO:teuthology.orchestra.run.smithi031.stdout:5325 trunc from 0xc804cc6 to 0xec59ced 2024-09-13T07:13:24.837 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:24.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:24.838 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:24.838 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:24.838 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:24.840 INFO:teuthology.orchestra.run.smithi031.stdout:5326 punch from 0x3eeeeb to 0x3f3a72, (0x4b87 bytes) 2024-09-13T07:13:24.840 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.840 INFO:teuthology.orchestra.run.smithi031.stdout:5329 writesame 0xaa54fd0 thru 0xaa5d176 data_size 0x1(0x81a7 bytes) 2024-09-13T07:13:24.847 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:24.855 INFO:teuthology.orchestra.run.smithi031.stdout:5333 read 0x449f12e thru 0x44ab30f (0xc1e2 bytes) 2024-09-13T07:13:24.855 INFO:teuthology.orchestra.run.smithi031.stdout:5334 read 0x7c9cca3 thru 0x7ca3544 (0x68a2 bytes) 2024-09-13T07:13:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:5335 punch from 0x5e9e334 to 0x5ea7f8c, (0x9c58 bytes) 2024-09-13T07:13:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:5336 read 0xc942a4c thru 0xc94eacc (0xc081 bytes) 2024-09-13T07:13:24.858 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:24.866 INFO:teuthology.orchestra.run.smithi031.stdout:5337 trunc from 0xec59ced to 0x644b03d 2024-09-13T07:13:24.877 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:24.878 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:24.887 INFO:teuthology.orchestra.run.smithi031.stdout:5338 writesame 0xb3a9cc0 thru 0xb3b15ad data_size 0x17(0x78ee bytes) 2024-09-13T07:13:24.887 INFO:teuthology.orchestra.run.smithi031.stdout:5339 punch from 0x1beba1c to 0x1bf5994, (0x9f78 bytes) 2024-09-13T07:13:24.887 INFO:teuthology.orchestra.run.smithi031.stdout:5340 read 0xd84074 thru 0xd93ff6 (0xff83 bytes) 2024-09-13T07:13:24.887 INFO:teuthology.orchestra.run.smithi031.stdout:5341 writesame 0x7fa1ddf thru 0x7fa8663 data_size 0x9(0x6885 bytes) 2024-09-13T07:13:24.888 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:24.889+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:24.891 INFO:teuthology.orchestra.run.smithi031.stdout:5342 writesame 0x5b4c873 thru 0x5b5c46b data_size 0x61(0xfbf9 bytes) 2024-09-13T07:13:24.895 INFO:teuthology.orchestra.run.smithi031.stdout:5343 read 0x5d139a7 thru 0x5d1638c (0x29e6 bytes) 2024-09-13T07:13:24.895 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:24.896 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:24.898 INFO:teuthology.orchestra.run.smithi031.stdout:5344 trunc from 0xb3b15ae to 0x856dc2c 2024-09-13T07:13:24.912 INFO:teuthology.orchestra.run.smithi031.stdout:5345 write 0x97b76e0 thru 0x97c54d0 (0xddf1 bytes) 2024-09-13T07:13:24.913 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:24.915 INFO:teuthology.orchestra.run.smithi031.stdout:5346 read 0x61ffa5d thru 0x6204eb4 (0x5458 bytes) 2024-09-13T07:13:24.915 INFO:teuthology.orchestra.run.smithi031.stdout:5347 read 0x1f67fc0 thru 0x1f6fb03 (0x7b44 bytes) 2024-09-13T07:13:24.916 INFO:teuthology.orchestra.run.smithi031.stdout:5348 writesame 0x3a7a419 thru 0x3a81640 data_size 0x1a(0x7228 bytes) 2024-09-13T07:13:24.919 INFO:teuthology.orchestra.run.smithi031.stdout:5349 write 0x607d66 thru 0x6096c8 (0x1963 bytes) 2024-09-13T07:13:24.920 INFO:teuthology.orchestra.run.smithi031.stdout:5351 read 0x76b6d17 thru 0x76be061 (0x734b bytes) 2024-09-13T07:13:24.920 INFO:teuthology.orchestra.run.smithi031.stdout:5352 read 0x659516c thru 0x65a27c5 (0xd65a bytes) 2024-09-13T07:13:24.921 INFO:teuthology.orchestra.run.smithi031.stdout:5353 read 0x4790a19 thru 0x4797501 (0x6ae9 bytes) 2024-09-13T07:13:24.921 INFO:teuthology.orchestra.run.smithi031.stdout:5354 write 0x23c8608 thru 0x23d7427 (0xee20 bytes) 2024-09-13T07:13:24.922 INFO:teuthology.orchestra.run.smithi031.stdout:5355 read 0x830a888 thru 0x830aa32 (0x1ab bytes) 2024-09-13T07:13:24.931 INFO:teuthology.orchestra.run.smithi031.stdout:5356 trunc from 0x97c54d1 to 0x2872bfa 2024-09-13T07:13:24.931 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:24.931 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:24.944 INFO:teuthology.orchestra.run.smithi031.stdout:5357 read 0x1d88a0c thru 0x1d8d814 (0x4e09 bytes) 2024-09-13T07:13:24.949 INFO:teuthology.orchestra.run.smithi031.stdout:5358 write 0x2a4fce9 thru 0x2a58040 (0x8358 bytes) 2024-09-13T07:13:24.950 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:24.950 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:24.952 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.964 INFO:teuthology.orchestra.run.smithi031.stdout:5360 write 0x7fd2998 thru 0x7fded3e (0xc3a7 bytes) 2024-09-13T07:13:24.966 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:24.967+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:24.967 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:24.967 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:24.969 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:24.969 INFO:teuthology.orchestra.run.smithi031.stdout:5362 read 0x246c8e8 thru 0x24723c5 (0x5ade bytes) 2024-09-13T07:13:24.969 INFO:teuthology.orchestra.run.smithi031.stdout:5364 write 0x533e4e2 thru 0x5340512 (0x2031 bytes) 2024-09-13T07:13:24.972 INFO:teuthology.orchestra.run.smithi031.stdout:5365 read 0x73af80e thru 0x73b4011 (0x4804 bytes) 2024-09-13T07:13:24.972 INFO:teuthology.orchestra.run.smithi031.stdout:5366 write 0x7821374 thru 0x782a177 (0x8e04 bytes) 2024-09-13T07:13:24.977 INFO:teuthology.orchestra.run.smithi031.stdout:5367 punch from 0x32696c7 to 0x326da3c, (0x4375 bytes) 2024-09-13T07:13:24.977 INFO:teuthology.orchestra.run.smithi031.stdout:5368 write 0x773085e thru 0x77393e9 (0x8b8c bytes) 2024-09-13T07:13:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:24.992 INFO:teuthology.orchestra.run.smithi031.stdout:5372 write 0xb387699 thru 0xb393860 (0xc1c8 bytes) 2024-09-13T07:13:24.996 INFO:teuthology.orchestra.run.smithi031.stdout:5373 read 0x88594d1 thru 0x885cbb2 (0x36e2 bytes) 2024-09-13T07:13:24.999 INFO:teuthology.orchestra.run.smithi031.stdout:5374 trunc from 0xb393861 to 0xe30702a 2024-09-13T07:13:24.999 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:24.999 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.004 INFO:teuthology.orchestra.run.smithi031.stdout:5375 write 0xef85a7 thru 0xef9811 (0x126b bytes) 2024-09-13T07:13:25.005 INFO:teuthology.orchestra.run.smithi031.stdout:5376 read 0x2f02de thru 0x2f6de5 (0x6b08 bytes) 2024-09-13T07:13:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:5377 write 0x1be5c66 thru 0x1be8fe5 (0x3380 bytes) 2024-09-13T07:13:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:5378 clone 30 order 19 su 8192 sc 6 2024-09-13T07:13:25.008 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:25.067 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:25.068+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:25.152 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:25.153 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.161 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:25.204 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:25.205+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:25.263 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.269 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:25.270+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:25.271 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:25.370 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:25.371+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:25.414 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:25.414 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:25.471 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:25.472+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:25.536 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:25.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:13:25.587 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:25.588+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:25.673 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:25.674+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:25.696 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:25.696 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:25.774 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:25.775+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:25.814 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:25.814 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:25.907 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:25.908+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:25.970 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:25.970 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:25.975 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:25.976+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:25.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:26.076 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:26.077+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:26.124 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:26.124 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:26.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:26.177 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:26.178+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:26.240 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:26.240 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:26.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:26.292 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:26.293+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:26.355 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:26.355 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:26.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:26.404 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:26.405+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:13:26.482 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:26.483+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:26.514 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:26.514 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:26.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:26.606 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:26.607+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:26.669 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:26.669 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:26.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:26.714 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T07:13:26.715 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:26.715+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:26.726 INFO:teuthology.orchestra.run.smithi031.stdout:5379 punch from 0xafe9549 to 0xaff0da4, (0x785b bytes) 2024-09-13T07:13:26.731 INFO:teuthology.orchestra.run.smithi031.stdout:5380 writesame 0x653ce7b thru 0x6541190 data_size 0x1f(0x4316 bytes) 2024-09-13T07:13:26.740 INFO:teuthology.orchestra.run.smithi031.stdout:5381 write 0xcd6d6a4 thru 0xcd77ef3 (0xa850 bytes) 2024-09-13T07:13:26.754 INFO:teuthology.orchestra.run.smithi031.stdout:5382 read 0x605a089 thru 0x605f4e9 (0x5461 bytes) 2024-09-13T07:13:26.754 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:26.754 INFO:teuthology.orchestra.run.smithi031.stdout:5384 read 0xb8f881f thru 0xb8fb5b3 (0x2d95 bytes) 2024-09-13T07:13:26.754 INFO:teuthology.orchestra.run.smithi031.stdout:5385 read 0xa1fd8f8 thru 0xa201294 (0x399d bytes) 2024-09-13T07:13:26.754 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:26.760 INFO:teuthology.orchestra.run.smithi031.stdout:5388 write 0xea2e3ae thru 0xea38a53 (0xa6a6 bytes) 2024-09-13T07:13:26.772 INFO:teuthology.orchestra.run.smithi031.stdout:5389 punch from 0xa32560f to 0xa32606e, (0xa5f bytes) 2024-09-13T07:13:26.772 INFO:teuthology.orchestra.run.smithi031.stdout:5392 read 0x802d1fc thru 0x802f6ed (0x24f2 bytes) 2024-09-13T07:13:26.772 INFO:teuthology.orchestra.run.smithi031.stdout:5394 writesame 0x93adedc thru 0x93b9bbc data_size 0x1(0xbce1 bytes) 2024-09-13T07:13:26.785 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:26.786+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:26.788 INFO:teuthology.orchestra.run.smithi031.stdout:5395 punch from 0x9dbb2a5 to 0x9dcaffe, (0xfd59 bytes) 2024-09-13T07:13:26.788 INFO:teuthology.orchestra.run.smithi031.stdout:5397 write 0x96cd376 thru 0x96da119 (0xcda4 bytes) 2024-09-13T07:13:26.799 INFO:teuthology.orchestra.run.smithi031.stdout:5398 trunc from 0xea38a54 to 0xc3103c5 2024-09-13T07:13:26.820 INFO:teuthology.orchestra.run.smithi031.stdout:5401 write 0x5037dca thru 0x50469b0 (0xebe7 bytes) 2024-09-13T07:13:26.833 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:26.833 INFO:teuthology.orchestra.run.smithi031.stdout:5405 write 0xa7cb5e thru 0xa85e1e (0x92c1 bytes) 2024-09-13T07:13:26.857 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:26.857 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:26.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:26.878 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:26.878 INFO:teuthology.orchestra.run.smithi031.stdout:5407 read 0x7e4ff21 thru 0x7e5f1ac (0xf28c bytes) 2024-09-13T07:13:26.880 INFO:teuthology.orchestra.run.smithi031.stdout:5408 punch from 0xb9477c8 to 0xb94f9f2, (0x822a bytes) 2024-09-13T07:13:26.881 INFO:teuthology.orchestra.run.smithi031.stdout:5409 write 0x9b899b8 thru 0x9b93d0d (0xa356 bytes) 2024-09-13T07:13:26.885 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:26.887+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:26.890 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:26.890 INFO:teuthology.orchestra.run.smithi031.stdout:5411 punch from 0x7d9c6c4 to 0x7da6085, (0x99c1 bytes) 2024-09-13T07:13:26.890 INFO:teuthology.orchestra.run.smithi031.stdout:5412 writesame 0x2cf424 thru 0x2d78c6 data_size 0x1(0x84a3 bytes) 2024-09-13T07:13:26.936 INFO:teuthology.orchestra.run.smithi031.stdout:5413 read 0x265b80e thru 0x265e9a3 (0x3196 bytes) 2024-09-13T07:13:26.940 INFO:teuthology.orchestra.run.smithi031.stdout:5414 trunc from 0xc3103c5 to 0x8f7d345 2024-09-13T07:13:26.973 INFO:teuthology.orchestra.run.smithi031.stdout:5415 write 0x27ab80b thru 0x27afde9 (0x45df bytes) 2024-09-13T07:13:26.987 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:26.988+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:26.987 INFO:teuthology.orchestra.run.smithi031.stdout:5416 write 0x8e4a477 thru 0x8e4a8a8 (0x432 bytes) 2024-09-13T07:13:26.989 INFO:teuthology.orchestra.run.smithi031.stdout:5418 read 0x15a59eb thru 0x15a5ca2 (0x2b8 bytes) 2024-09-13T07:13:26.992 INFO:teuthology.orchestra.run.smithi031.stdout:5421 trunc from 0x8f7d345 to 0xbc9427d 2024-09-13T07:13:26.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:27.000 INFO:teuthology.orchestra.run.smithi031.stdout:5423 writesame 0x675c196 thru 0x6766102 data_size 0x1(0x9f6d bytes) 2024-09-13T07:13:27.006 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:27.007 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:27.010 INFO:teuthology.orchestra.run.smithi031.stdout:5424 write 0xb8a06a5 thru 0xb8a08a0 (0x1fc bytes) 2024-09-13T07:13:27.013 INFO:teuthology.orchestra.run.smithi031.stdout:5425 write 0x66f305 thru 0x66fa67 (0x763 bytes) 2024-09-13T07:13:27.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:27.031 INFO:teuthology.orchestra.run.smithi031.stdout:5427 write 0xeb7687e thru 0xeb8264e (0xbdd1 bytes) 2024-09-13T07:13:27.037 INFO:teuthology.orchestra.run.smithi031.stdout:5428 read 0x533c55d thru 0x5347ba5 (0xb649 bytes) 2024-09-13T07:13:27.039 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:27.040 INFO:teuthology.orchestra.run.smithi031.stdout:5430 trunc from 0xeb8264f to 0xdef982e 2024-09-13T07:13:27.051 INFO:teuthology.orchestra.run.smithi031.stdout:5432 read 0xbd89058 thru 0xbd8d9d6 (0x497f bytes) 2024-09-13T07:13:27.051 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:27.056 INFO:teuthology.orchestra.run.smithi031.stdout:5434 trunc from 0xdef982e to 0x9215df4 2024-09-13T07:13:27.066 INFO:teuthology.orchestra.run.smithi031.stdout:5437 writesame 0x1f085f9 thru 0x1f13d70 data_size 0x98(0xb778 bytes) 2024-09-13T07:13:27.088 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:27.089+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:27.115 INFO:teuthology.orchestra.run.smithi031.stdout:5439 write 0x45158b7 thru 0x451891f (0x3069 bytes) 2024-09-13T07:13:27.123 INFO:teuthology.orchestra.run.smithi031.stdout:5440 clone 31 order 23 su 4194304 sc 12 2024-09-13T07:13:27.138 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:27.139 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:27.146 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:27.146 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:27.155 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:27.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:27.156 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:27.172 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:27.187 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:27.199 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:27.200+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:27.204 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:27.204 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:27.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:27.220 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:27.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:27.237 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:27.253 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:27.259 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:27.259 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:27.259 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:27.259 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:27.268 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:27.309 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:27.310+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:27.390 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:27.391+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:27.412 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:27.413 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:27.413 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:27.413 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:27.413 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:27.418 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:27.491 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:27.492+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone30 2024-09-13T07:13:27.549 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:27.549 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:27.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:27.549 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:27.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.549 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.332840144999999, 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:13:27.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:13:27.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:13:27.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52442 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3985408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.959024209999999, 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040126756, 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.556 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52442 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:27.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590088+0000", 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590093+0000", 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0799999999999996e-06 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.558 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590096+0000", 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2679999999999999e-06 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590099+0000", 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.481e-06 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630022+0000", 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039923472000000002 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630106+0000", 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4264999999999994e-05 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630215+0000", 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000108399 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52443 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.560 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.959011299, 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040282897999999998, 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52443 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:27.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590101+0000", 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590102+0000", 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2130000000000001e-06 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590105+0000", 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.881e-06 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590107+0000", 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6510000000000001e-06 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630253+0000", 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040145713999999999 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630327+0000", 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3813000000000006e-05 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630384+0000", 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6842999999999998e-05 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.564 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52444 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4034560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.959002097999999, 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040488365999999998, 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52444 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590110+0000", 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590111+0000", 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.176e-06 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590114+0000", 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7889999999999999e-06 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590116+0000", 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.435e-06 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630413+0000", 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040296998000000001 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630493+0000", 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9610999999999994e-05 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630598+0000", 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000105566 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52445 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4059136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.958994006000001, 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040607740000000003, 2024-09-13T07:13:27.569 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52445 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590118+0000", 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590119+0000", 2024-09-13T07:13:27.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.288e-06 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590122+0000", 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7030000000000002e-06 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590124+0000", 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4250000000000001e-06 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630628+0000", 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040503577999999998 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630671+0000", 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3458999999999998e-05 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630726+0000", 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4490000000000002e-05 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52446 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4083712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.958985860999999, 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040820130000000003, 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52446 2024-09-13T07:13:27.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.575 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590126+0000", 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590127+0000", 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.271e-06 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590130+0000", 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7389999999999999e-06 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590132+0000", 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630757+0000", 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040624614000000003 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630836+0000", 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8913999999999997e-05 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630946+0000", 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000110398 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52447 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4108288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.95897759, 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041064639, 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.578 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52447 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590135+0000", 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.579 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590134+0000", 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590136+0000", 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.266e-06 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590138+0000", 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.869e-06 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590141+0000", 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6699999999999998e-06 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.630984+0000", 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040843276999999997 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631072+0000", 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7386000000000007e-05 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631199+0000", 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012740000000000001 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52448 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4132864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.958968929000001, 2024-09-13T07:13:27.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041275991999999997, 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52448 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590143+0000", 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590144+0000", 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2109999999999999e-06 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590147+0000", 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8820000000000001e-06 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590150+0000", 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631231+0000", 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041081307999999997 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631298+0000", 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7340000000000002e-05 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631419+0000", 2024-09-13T07:13:27.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000120986 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52449 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4157440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.958960269999999, 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041487405999999998, 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.588 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52449 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590152+0000", 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590151+0000", 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590153+0000", 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590156+0000", 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.807e-06 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590158+0000", 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.458e-06 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631453+0000", 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041294850000000001 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631534+0000", 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0908999999999994e-05 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631639+0000", 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000105099 2024-09-13T07:13:27.592 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:52450 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 4182016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.958934571, 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041663446999999999, 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.593 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 52450 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590178+0000", 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590177+0000", 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590179+0000", 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.705e-06 2024-09-13T07:13:27.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590182+0000", 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3689999999999998e-06 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.590185+0000", 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631666+0000", 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041480994 2024-09-13T07:13:27.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631741+0000", 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4666999999999995e-05 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.631841+0000", 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000100449 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53407 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3158016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.856979547999998, 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040141932999999998, 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53407 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692133+0000", 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692132+0000", 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692134+0000", 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3080000000000002e-06 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692140+0000", 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8080000000000001e-06 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692145+0000", 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732076+0000", 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.039931543 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732162+0000", 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5657999999999999e-05 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732275+0000", 2024-09-13T07:13:27.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00011256600000000001 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53408 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3182592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.856963496999999, 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040246709999999998, 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.602 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53408 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692149+0000", 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692148+0000", 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692150+0000", 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.877e-06 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692156+0000", 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.015e-06 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692161+0000", 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.544e-06 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732301+0000", 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040140051000000003 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732342+0000", 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1325999999999997e-05 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732395+0000", 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3229999999999997e-05 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53409 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3207168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.856938289999999, 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040446375, 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53409 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692174+0000", 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692173+0000", 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692180+0000", 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2620000000000004e-06 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692186+0000", 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.861e-06 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692191+0000", 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732426+0000", 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040234501999999998 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732513+0000", 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6991999999999997e-05 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732620+0000", 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000107413 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53410 3.f 3:f6d12159:::rbd_data.1173fee7d7bf.0000000000000009:15 [read 3231744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.856916748, 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040545431999999999, 2024-09-13T07:13:27.610 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53410 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692195+0000", 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692197+0000", 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1280000000000002e-06 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692203+0000", 2024-09-13T07:13:27.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8080000000000001e-06 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.692208+0000", 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2399999999999998e-06 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732647+0000", 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040438395000000002 2024-09-13T07:13:27.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732686+0000", 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9573000000000002e-05 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.732741+0000", 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4731000000000002e-05 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53421 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.855059357999998, 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040071946999999997, 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53421 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694053+0000", 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694052+0000", 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694058+0000", 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7679999999999997e-06 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694064+0000", 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2700000000000001e-06 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694069+0000", 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734115+0000", 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040045348000000001 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734125+0000", 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0151e-05 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53425 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:27.618 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.854975176, 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040069779, 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53425 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694137+0000", 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694139+0000", 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1330000000000002e-06 2024-09-13T07:13:27.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694145+0000", 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9669999999999998e-06 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694149+0000", 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5680000000000001e-06 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734197+0000", 2024-09-13T07:13:27.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040048148999999998 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734207+0000", 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.3670000000000003e-06 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:53427 3.f 3.9a848b6f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.854942469000001, 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040080377, 2024-09-13T07:13:27.622 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 53427 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694170+0000", 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694169+0000", 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694172+0000", 2024-09-13T07:13:27.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2359999999999999e-06 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694177+0000", 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8150000000000002e-06 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.694182+0000", 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6380000000000003e-06 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734242+0000", 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040059715000000003 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.734250+0000", 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3429999999999996e-06 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:71808 3.a e50/19 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head v 50'527, mlcod=50'525)", 2024-09-13T07:13:27.626 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.964733935000002, 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041667003000000001, 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584378+0000", 2024-09-13T07:13:27.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584377+0000", 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584385+0000", 2024-09-13T07:13:27.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2949999999999994e-06 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584402+0000", 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6458999999999999e-05 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584409+0000", 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584457+0000", 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8130000000000002e-05 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584475+0000", 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7677e-05 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626008+0000", 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041532931000000002 2024-09-13T07:13:27.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626045+0000", 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7654999999999998e-05 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71817 3.f 3:f1787d65:::rbd_object_map.1173ab534c79:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 17.963841849000001, 2024-09-13T07:13:27.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040930686000000001, 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71817 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585270+0000", 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585269+0000", 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585275+0000", 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.237e-06 2024-09-13T07:13:27.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585284+0000", 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.0529999999999996e-06 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585288+0000", 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5029999999999999e-06 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585318+0000", 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0045000000000001e-05 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.585364+0000", 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5985000000000001e-05 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626068+0000", 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.040703678 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626114+0000", 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6508999999999998e-05 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626201+0000", 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6655000000000003e-05 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3:f56afaab:::rbd_object_map.117375db170a:head [read 0~112] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 13.202516886, 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0257515829999999, 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346593+0000", 2024-09-13T07:13:27.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346603+0000", 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0037999999999999e-05 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346626+0000", 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3169000000000001e-05 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346638+0000", 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1712999999999999e-05 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346719+0000", 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0980000000000001e-05 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346730+0000", 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0657e-05 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348732+0000", 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020022690000000001 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348743+0000", 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0431999999999999e-05 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.371984+0000", 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0232416790000001 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372159+0000", 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017503500000000001 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372347+0000", 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018757299999999999 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.642 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:27.644 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:27.644 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:27.644+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:27.744 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:27.745+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:27.821 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:27.821 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:27.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.821 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96323 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.821 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813033+0000", 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0029222950000000001, 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0029421669999999999, 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96323 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813033+0000", 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813033+0000", 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813032+0000", 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813035+0000", 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4339999999999999e-06 2024-09-13T07:13:27.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813041+0000", 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6960000000000002e-06 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813045+0000", 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3889999999999998e-06 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96354 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813552+0000", 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0024032369999999999, 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025038740000000001, 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96354 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813552+0000", 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813552+0000", 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813551+0000", 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813557+0000", 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3959999999999997e-06 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813561+0000", 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9199999999999997e-06 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813563+0000", 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.65e-06 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96388 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815697+0000", 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00025763799999999998, 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00041349300000000002, 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96388 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815697+0000", 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815697+0000", 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815697+0000", 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815699+0000", 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.906e-06 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815704+0000", 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7060000000000003e-06 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815707+0000", 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8639999999999998e-06 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.830 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96324 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813049+0000", 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0029061320000000001, 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003117758, 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.831 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96324 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813049+0000", 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813049+0000", 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813049+0000", 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813051+0000", 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1550000000000001e-06 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813056+0000", 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6790000000000002e-06 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813061+0000", 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4660000000000001e-06 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96355 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813567+0000", 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0023879359999999998, 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0026838880000000002, 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.834 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96355 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813567+0000", 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813567+0000", 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813567+0000", 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813569+0000", 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9530000000000002e-06 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813575+0000", 2024-09-13T07:13:27.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8719999999999996e-06 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813580+0000", 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7990000000000001e-06 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96389 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815711+0000", 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00024427900000000002, 2024-09-13T07:13:27.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00059302900000000002, 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96389 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815711+0000", 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815711+0000", 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815710+0000", 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815712+0000", 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8929999999999999e-06 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815717+0000", 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815721+0000", 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9949999999999999e-06 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.840 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96325 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813064+0000", 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0028907170000000001, 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032830099999999998, 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96325 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813064+0000", 2024-09-13T07:13:27.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813064+0000", 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813064+0000", 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813066+0000", 2024-09-13T07:13:27.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0760000000000001e-06 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813071+0000", 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.366e-06 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813076+0000", 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1849999999999997e-06 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96356 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813584+0000", 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0023707960000000001, 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002810927, 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96356 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813584+0000", 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813584+0000", 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813584+0000", 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813586+0000", 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1280000000000002e-06 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813591+0000", 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.237e-06 2024-09-13T07:13:27.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813597+0000", 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3940000000000004e-06 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96390 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815724+0000", 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00023063200000000001, 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00070800300000000002, 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.847 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96390 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815724+0000", 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815724+0000", 2024-09-13T07:13:27.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815724+0000", 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815726+0000", 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8980000000000001e-06 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815731+0000", 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8409999999999999e-06 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815734+0000", 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6179999999999999e-06 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96326 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813079+0000", 2024-09-13T07:13:27.850 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0028757050000000001, 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034079610000000001, 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96326 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813079+0000", 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813079+0000", 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813079+0000", 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813081+0000", 2024-09-13T07:13:27.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813087+0000", 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.806e-06 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813091+0000", 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.216e-06 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.853 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96361 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.814938+0000", 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.001017261, 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015994819999999999, 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96361 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.854 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.814938+0000", 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.814938+0000", 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.814937+0000", 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:27.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.814942+0000", 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6799999999999998e-06 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.814972+0000", 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9386e-05 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.814986+0000", 2024-09-13T07:13:27.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4389000000000001e-05 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96391 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815762+0000", 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00019286800000000001, 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00082132199999999998, 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96391 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815762+0000", 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815762+0000", 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815762+0000", 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815770+0000", 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4589999999999998e-06 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815780+0000", 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.7869999999999996e-06 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815787+0000", 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7340000000000002e-06 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96327 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813136+0000", 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0028188029999999999, 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035008000000000001, 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96327 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813136+0000", 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813136+0000", 2024-09-13T07:13:27.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813136+0000", 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813140+0000", 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6659999999999999e-06 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813154+0000", 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4304000000000001e-05 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813161+0000", 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8789999999999997e-06 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96362 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.863 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815001+0000", 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.000954104, 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00169184, 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96362 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815001+0000", 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815001+0000", 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815000+0000", 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815009+0000", 2024-09-13T07:13:27.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5140000000000001e-06 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815022+0000", 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3360000000000001e-05 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815028+0000", 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2950000000000001e-06 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96392 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815792+0000", 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00016310399999999999, 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00095394100000000003, 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96392 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815792+0000", 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815792+0000", 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815791+0000", 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815794+0000", 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7470000000000001e-06 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815806+0000", 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1963e-05 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815812+0000", 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7230000000000001e-06 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.869 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96363 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815039+0000", 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00091613600000000001, 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0017608470000000001, 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96363 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815039+0000", 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815039+0000", 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815039+0000", 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815048+0000", 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5410000000000006e-06 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815057+0000", 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7450000000000001e-06 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815062+0000", 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2569999999999998e-06 2024-09-13T07:13:27.872 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96393 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815817+0000", 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00013775699999999999, 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0010402510000000001, 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.873 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96393 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815817+0000", 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815817+0000", 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815817+0000", 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815819+0000", 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.933e-06 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815822+0000", 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6459999999999999e-06 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815825+0000", 2024-09-13T07:13:27.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7379999999999999e-06 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96328 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813167+0000", 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0027878489999999998, 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037496690000000002, 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.876 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96328 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813167+0000", 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813167+0000", 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813167+0000", 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813169+0000", 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5299999999999999e-06 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813175+0000", 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813182+0000", 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8519999999999998e-06 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96364 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815074+0000", 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00088115600000000002, 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0018928969999999999, 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.879 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96364 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815074+0000", 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815074+0000", 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815066+0000", 2024-09-13T07:13:27.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999928 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815076+0000", 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9930000000000007e-06 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815087+0000", 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0532e-05 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815092+0000", 2024-09-13T07:13:27.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9540000000000003e-06 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96394 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815827+0000", 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.000127739, 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0011710290000000001, 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96394 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.882 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815827+0000", 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815827+0000", 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815827+0000", 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815828+0000", 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3710000000000001e-06 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815831+0000", 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815834+0000", 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.481e-06 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.884 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96329 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813188+0000", 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0027672529999999999, 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038372789999999999, 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96329 2024-09-13T07:13:27.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813188+0000", 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813188+0000", 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813187+0000", 2024-09-13T07:13:27.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813190+0000", 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6429999999999999e-06 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813195+0000", 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0669999999999999e-06 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813198+0000", 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8329999999999998e-06 2024-09-13T07:13:27.887 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96365 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815143+0000", 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00081174700000000003, 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019082770000000001, 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96365 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.888 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815143+0000", 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815143+0000", 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815143+0000", 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815146+0000", 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5470000000000001e-06 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815159+0000", 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2594e-05 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815165+0000", 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7769999999999999e-06 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96395 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815836+0000", 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00011912400000000001, 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001259643, 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96395 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815836+0000", 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815836+0000", 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815836+0000", 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815837+0000", 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.24e-06 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815840+0000", 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9289999999999998e-06 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815842+0000", 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4600000000000002e-06 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.893 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96330 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813200+0000", 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.002754884, 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039466240000000001, 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96330 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813200+0000", 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813200+0000", 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813200+0000", 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813205+0000", 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5110000000000003e-06 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813209+0000", 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2499999999999998e-06 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813211+0000", 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.43e-06 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96366 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815177+0000", 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00077772900000000001, 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.002018022, 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96366 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815177+0000", 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815177+0000", 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815177+0000", 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815182+0000", 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815188+0000", 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.409e-06 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815193+0000", 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8459999999999999e-06 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96396 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815844+0000", 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.000110541, 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001402627, 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96396 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815844+0000", 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815844+0000", 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815844+0000", 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815845+0000", 2024-09-13T07:13:27.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1909999999999999e-06 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815848+0000", 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.83e-06 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815851+0000", 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5110000000000002e-06 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.902 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96331 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813214+0000", 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0027406150000000001, 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040699270000000001, 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96331 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.903 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813214+0000", 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813214+0000", 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813214+0000", 2024-09-13T07:13:27.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813216+0000", 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3960000000000001e-06 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813219+0000", 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7799999999999998e-06 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813222+0000", 2024-09-13T07:13:27.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9869999999999999e-06 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96367 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815202+0000", 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00075343899999999995, 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021198900000000001, 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.906 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96367 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815202+0000", 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815202+0000", 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815201+0000", 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815204+0000", 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7e-06 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815210+0000", 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0650000000000004e-06 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815215+0000", 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7779999999999998e-06 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96397 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815853+0000", 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00010187999999999999, 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00149577, 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.909 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96397 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815853+0000", 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815853+0000", 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815853+0000", 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815854+0000", 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.35e-06 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815857+0000", 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7779999999999999e-06 2024-09-13T07:13:27.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815859+0000", 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4439999999999998e-06 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96332 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813224+0000", 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0027305739999999999, 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004151437, 2024-09-13T07:13:27.912 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96332 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813224+0000", 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813224+0000", 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813224+0000", 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813226+0000", 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3540000000000001e-06 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813228+0000", 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8559999999999998e-06 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813231+0000", 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6800000000000002e-06 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96398 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815875+0000", 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.0436e-05, 2024-09-13T07:13:27.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0015412290000000001, 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96398 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815875+0000", 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815875+0000", 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815874+0000", 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815876+0000", 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8679999999999999e-06 2024-09-13T07:13:27.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815887+0000", 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.079e-05 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815891+0000", 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2729999999999996e-06 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.918 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96333 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813233+0000", 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0027217489999999999, 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0042362479999999997, 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96333 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813233+0000", 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813233+0000", 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813233+0000", 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813235+0000", 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5269999999999999e-06 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813237+0000", 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813240+0000", 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.345e-06 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.921 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96399 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815895+0000", 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.9667000000000003e-05, 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.001627215, 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96399 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.922 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815895+0000", 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815895+0000", 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815895+0000", 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815897+0000", 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3099999999999999e-06 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815902+0000", 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.074e-06 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815906+0000", 2024-09-13T07:13:27.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.985e-06 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96334 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813242+0000", 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0027132290000000002, 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0043354090000000001, 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96334 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813242+0000", 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813242+0000", 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813242+0000", 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813243+0000", 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.249e-06 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813246+0000", 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8100000000000002e-06 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813248+0000", 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3470000000000001e-06 2024-09-13T07:13:27.927 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96368 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815379+0000", 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00057639599999999996, 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0022652509999999998, 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96368 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.928 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815379+0000", 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815379+0000", 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815378+0000", 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815382+0000", 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6540000000000001e-06 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815390+0000", 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815396+0000", 2024-09-13T07:13:27.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0769999999999996e-06 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96400 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815910+0000", 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.5513000000000003e-05, 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0017993410000000001, 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96400 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815910+0000", 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815910+0000", 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815909+0000", 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815911+0000", 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.733e-06 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815917+0000", 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:13:27.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815921+0000", 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9369999999999997e-06 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96335 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813250+0000", 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.002704985, 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0045126569999999998, 2024-09-13T07:13:27.934 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96335 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813250+0000", 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813250+0000", 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813250+0000", 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813251+0000", 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3170000000000001e-06 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813257+0000", 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8680000000000004e-06 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813259+0000", 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4339999999999999e-06 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96369 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.937 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815401+0000", 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00055380900000000005, 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0024164859999999998, 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96369 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815401+0000", 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815401+0000", 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815401+0000", 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815404+0000", 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8289999999999999e-06 2024-09-13T07:13:27.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815410+0000", 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815414+0000", 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5460000000000002e-06 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96401 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815924+0000", 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 3.1401e-05, 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0019495249999999999, 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96401 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815924+0000", 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815924+0000", 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815923+0000", 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815925+0000", 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.131e-06 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815930+0000", 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8570000000000003e-06 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815934+0000", 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6519999999999999e-06 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.943 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96336 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813261+0000", 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.002693647, 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.004676838, 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96336 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813261+0000", 2024-09-13T07:13:27.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813261+0000", 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813261+0000", 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813262+0000", 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.248e-06 2024-09-13T07:13:27.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813265+0000", 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.79e-06 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813268+0000", 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7360000000000001e-06 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.946 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96370 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815418+0000", 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00053685600000000005, 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0025744779999999998, 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96370 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.947 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815418+0000", 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815418+0000", 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815418+0000", 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815420+0000", 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9989999999999998e-06 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815426+0000", 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7799999999999997e-06 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815430+0000", 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.51e-06 2024-09-13T07:13:27.949 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96402 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815937+0000", 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.7808999999999998e-05, 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0021091870000000002, 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96402 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815937+0000", 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815937+0000", 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815937+0000", 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815939+0000", 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.672e-06 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815943+0000", 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7190000000000001e-06 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815948+0000", 2024-09-13T07:13:27.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1679999999999997e-06 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96337 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813271+0000", 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0026837670000000001, 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0048323639999999996, 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.953 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96337 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813271+0000", 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813271+0000", 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813271+0000", 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813273+0000", 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7129999999999999e-06 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813278+0000", 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0889999999999999e-06 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813281+0000", 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2739999999999999e-06 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96371 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815434+0000", 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00052072299999999995, 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0027241399999999999, 2024-09-13T07:13:27.956 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96371 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815434+0000", 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815434+0000", 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815434+0000", 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815436+0000", 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3290000000000001e-06 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815442+0000", 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.755e-06 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815446+0000", 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2969999999999997e-06 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:96405 3.d e66/56 3:b178bf22:::rbd_data.11738a001609.00000000000000f2:head v 66'246, mlcod=66'243)", 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.817459+0000", 2024-09-13T07:13:27.959 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4294967295.9984965, 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00076030399999999995, 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817459+0000", 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817459+0000", 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817458+0000", 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817477+0000", 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9216e-05 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817491+0000", 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.397e-05 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817498+0000", 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0360000000000001e-06 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817561+0000", 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3519e-05 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.817581+0000", 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9388000000000001e-05 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96338 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813283+0000", 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0026717809999999998, 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0049943019999999999, 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96338 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813283+0000", 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813283+0000", 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813283+0000", 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813284+0000", 2024-09-13T07:13:27.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3519999999999999e-06 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813288+0000", 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.151e-06 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813291+0000", 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8859999999999997e-06 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96372 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815450+0000", 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00050485000000000005, 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0028824850000000002, 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96372 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815450+0000", 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815450+0000", 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815450+0000", 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815452+0000", 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1670000000000002e-06 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815457+0000", 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5219999999999998e-06 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815462+0000", 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5820000000000003e-06 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_repop(client.4467.0:96406 3.d e66/56)", 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.818005+0000", 2024-09-13T07:13:27.968 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4294967295.9979506, 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00038081299999999997, 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.818005+0000", 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.818005+0000", 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.818004+0000", 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.818010+0000", 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0090000000000004e-06 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.818020+0000", 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5910000000000002e-06 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.818026+0000", 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0379999999999996e-06 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96339 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813295+0000", 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.002659678, 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0051409719999999997, 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.971 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96339 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813295+0000", 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813295+0000", 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813295+0000", 2024-09-13T07:13:27.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813297+0000", 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1150000000000001e-06 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813305+0000", 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.055e-06 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813310+0000", 2024-09-13T07:13:27.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9509999999999997e-06 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96373 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815467+0000", 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00048846200000000005, 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030269540000000001, 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96373 2024-09-13T07:13:27.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815467+0000", 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815467+0000", 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815466+0000", 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815469+0000", 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6840000000000001e-06 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815475+0000", 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.613e-06 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815480+0000", 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5619999999999997e-06 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.976 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96340 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813315+0000", 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.002640482, 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0052490820000000004, 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96340 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.977 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813315+0000", 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813315+0000", 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813314+0000", 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813316+0000", 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2259999999999999e-06 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813324+0000", 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.011e-06 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813330+0000", 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.84e-06 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96374 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815484+0000", 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.000471409, 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0031344939999999998, 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96374 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815484+0000", 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815484+0000", 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815483+0000", 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815485+0000", 2024-09-13T07:13:27.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1390000000000002e-06 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815491+0000", 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2669999999999997e-06 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815495+0000", 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3699999999999997e-06 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96341 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.982 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813361+0000", 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0025935979999999999, 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0053173939999999996, 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96341 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813361+0000", 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813361+0000", 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813361+0000", 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813364+0000", 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8210000000000002e-06 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813372+0000", 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.5709999999999998e-06 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813378+0000", 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4369999999999998e-06 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96375 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815499+0000", 2024-09-13T07:13:27.985 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00045640999999999998, 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0032329630000000002, 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96375 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815499+0000", 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815499+0000", 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815498+0000", 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815500+0000", 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1490000000000001e-06 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815505+0000", 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6439999999999999e-06 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815509+0000", 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7500000000000001e-06 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96311 3.1c 3:3eabac8b:::rbd_object_map.11738a001609:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.807535+0000", 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0084195680000000005, 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.011235873, 2024-09-13T07:13:27.988 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96311 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.807535+0000", 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.807535+0000", 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.807535+0000", 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.807538+0000", 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.326e-06 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.807544+0000", 2024-09-13T07:13:27.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9379999999999997e-06 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.807550+0000", 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.811955+0000", 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0044051369999999999 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.812035+0000", 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9264999999999994e-05 2024-09-13T07:13:27.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.812402+0000", 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000367153 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96342 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813384+0000", 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0025714010000000001, 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054359430000000004, 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.992 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96342 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813384+0000", 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813384+0000", 2024-09-13T07:13:27.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813383+0000", 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813390+0000", 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7730000000000002e-06 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813397+0000", 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2219999999999996e-06 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813408+0000", 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1314e-05 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96376 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815512+0000", 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00044320999999999998, 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033370019999999999, 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.995 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96376 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815512+0000", 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815512+0000", 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815512+0000", 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815513+0000", 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8619999999999999e-06 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815518+0000", 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6990000000000002e-06 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815522+0000", 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6950000000000002e-06 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96343 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813414+0000", 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0025413200000000001, 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0054630210000000002, 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:27.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96343 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813414+0000", 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813414+0000", 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:27.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813413+0000", 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813416+0000", 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4439999999999998e-06 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813422+0000", 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4470000000000003e-06 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813429+0000", 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.409e-06 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96377 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815526+0000", 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00042942000000000002, 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0033924129999999999, 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96377 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815526+0000", 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815526+0000", 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815525+0000", 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815527+0000", 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.869e-06 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815532+0000", 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6859999999999997e-06 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815537+0000", 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96344 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813433+0000", 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0025221750000000002, 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0055238900000000001, 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.004 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96344 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813433+0000", 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813433+0000", 2024-09-13T07:13:28.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813433+0000", 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813434+0000", 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9209999999999999e-06 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813442+0000", 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:13:28.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813446+0000", 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4109999999999998e-06 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96378 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815540+0000", 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00041519199999999999, 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034598939999999998, 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.007 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96378 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815540+0000", 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815540+0000", 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815540+0000", 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815541+0000", 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8950000000000001e-06 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815547+0000", 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3199999999999999e-06 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815551+0000", 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.749e-06 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96345 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813450+0000", 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0025045610000000002, 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005584309, 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96345 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813450+0000", 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813450+0000", 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813450+0000", 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813452+0000", 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0999999999999998e-06 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813458+0000", 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7880000000000003e-06 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813463+0000", 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96379 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815554+0000", 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00040100500000000001, 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035195220000000002, 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.013 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96379 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815554+0000", 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815554+0000", 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815554+0000", 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815560+0000", 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.336e-06 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815565+0000", 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1540000000000001e-06 2024-09-13T07:13:28.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815569+0000", 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9779999999999999e-06 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96346 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813467+0000", 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0024875800000000001, 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0056402869999999999, 2024-09-13T07:13:28.016 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96346 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813467+0000", 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813467+0000", 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813467+0000", 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813469+0000", 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.272e-06 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813475+0000", 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0349999999999998e-06 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813481+0000", 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8270000000000003e-06 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96380 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815573+0000", 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00038227200000000002, 2024-09-13T07:13:28.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0035745579999999998, 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96380 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815573+0000", 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815573+0000", 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815572+0000", 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815574+0000", 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9309999999999998e-06 2024-09-13T07:13:28.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815579+0000", 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6789999999999996e-06 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815583+0000", 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.878e-06 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96347 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.022 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813485+0000", 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.002469649, 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0057075980000000004, 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96347 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813485+0000", 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813485+0000", 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813485+0000", 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813487+0000", 2024-09-13T07:13:28.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9269999999999999e-06 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813491+0000", 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1649999999999999e-06 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813494+0000", 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6979999999999998e-06 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96381 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815586+0000", 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00036892000000000001, 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0036601260000000001, 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96381 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815586+0000", 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815586+0000", 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815586+0000", 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815588+0000", 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.877e-06 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815593+0000", 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0810000000000001e-06 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815596+0000", 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6200000000000001e-06 2024-09-13T07:13:28.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96348 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813496+0000", 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0024586959999999998, 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0058077880000000004, 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96348 2024-09-13T07:13:28.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813496+0000", 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813496+0000", 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813496+0000", 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813497+0000", 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3510000000000001e-06 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813500+0000", 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813503+0000", 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9040000000000001e-06 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96382 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815600+0000", 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.000355213, 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0037562899999999998, 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96382 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815600+0000", 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815600+0000", 2024-09-13T07:13:28.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815599+0000", 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815602+0000", 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.334e-06 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815607+0000", 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.809e-06 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815611+0000", 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0010000000000003e-06 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96349 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813506+0000", 2024-09-13T07:13:28.035 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0024494529999999999, 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.005948325, 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96349 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813506+0000", 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813506+0000", 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813505+0000", 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813507+0000", 2024-09-13T07:13:28.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.299e-06 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813510+0000", 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6540000000000001e-06 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813513+0000", 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.559e-06 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96383 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815614+0000", 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.000341176, 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0038683799999999998, 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96383 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.039 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815614+0000", 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815614+0000", 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815614+0000", 2024-09-13T07:13:28.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815615+0000", 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9089999999999998e-06 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815620+0000", 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8890000000000001e-06 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815625+0000", 2024-09-13T07:13:28.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4279999999999998e-06 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96350 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813515+0000", 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0024401140000000002, 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0059991819999999996, 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96350 2024-09-13T07:13:28.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813515+0000", 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813515+0000", 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813515+0000", 2024-09-13T07:13:28.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813516+0000", 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1710000000000001e-06 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813520+0000", 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6389999999999998e-06 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813522+0000", 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3590000000000002e-06 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96384 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815637+0000", 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00031755599999999998, 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0039247630000000004, 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.045 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96384 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815637+0000", 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815637+0000", 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815637+0000", 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815639+0000", 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815649+0000", 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5950000000000005e-06 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815653+0000", 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2780000000000004e-06 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96351 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813524+0000", 2024-09-13T07:13:28.048 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.002431122, 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0060926979999999997, 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96351 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813524+0000", 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813524+0000", 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813524+0000", 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813525+0000", 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2449999999999999e-06 2024-09-13T07:13:28.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813528+0000", 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813530+0000", 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3779999999999999e-06 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96385 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815657+0000", 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00029806199999999998, 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0040167019999999996, 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96385 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815657+0000", 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815657+0000", 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815657+0000", 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815659+0000", 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1670000000000002e-06 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815664+0000", 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.916e-06 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815668+0000", 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8460000000000001e-06 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.054 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96352 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813532+0000", 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0024227469999999998, 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0061984409999999998, 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96352 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.055 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813532+0000", 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813532+0000", 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813532+0000", 2024-09-13T07:13:28.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813533+0000", 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1459999999999999e-06 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813537+0000", 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7929999999999999e-06 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813539+0000", 2024-09-13T07:13:28.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4729999999999999e-06 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96386 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815671+0000", 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00028431600000000001, 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0041130029999999996, 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.058 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96386 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815671+0000", 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815671+0000", 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815670+0000", 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815672+0000", 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.747e-06 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815677+0000", 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.702e-06 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815681+0000", 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7409999999999998e-06 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96353 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.813542+0000", 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.0024130649999999998, 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0062722519999999999, 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96353 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813542+0000", 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813542+0000", 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813542+0000", 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813543+0000", 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2190000000000001e-06 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813547+0000", 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6440000000000002e-06 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.813549+0000", 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4279999999999999e-06 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96387 3.1c 3.bc84f35c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.815684+0000", 2024-09-13T07:13:28.064 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.00027107299999999998, 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00415877, 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "queued for pg", 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96387 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815684+0000", 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815684+0000", 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815684+0000", 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815686+0000", 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8989999999999999e-06 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815690+0000", 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7559999999999999e-06 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.815694+0000", 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7239999999999998e-06 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: ], 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 79 2024-09-13T07:13:28.067 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:28.069 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:28.069 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:28.070+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:28.170 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:28.171+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:28.212 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:28.212 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:28.212 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:28.212 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:28.212 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:28.221 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:28.305 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:28.306+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:28.366 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:28.366 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:28.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:28.366 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:28.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.366 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.702417000000001, 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.371 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.702379349000001, 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:28.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:28.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.702360122000002, 2024-09-13T07:13:28.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:28.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:28.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:28.380 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.826286710000002, 2024-09-13T07:13:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:28.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:28.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96967 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:28.387 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51734938799999997, 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028701085000000001, 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96967 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848739+0000", 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848741+0000", 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.153e-06 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848743+0000", 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.723e-06 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848746+0000", 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.858e-06 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877369+0000", 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028623282999999999 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877406+0000", 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6139000000000001e-05 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877441+0000", 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151000000000003e-05 2024-09-13T07:13:28.391 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96968 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51734082699999995, 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028858604999999999, 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.392 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96968 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848749+0000", 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.091e-06 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848752+0000", 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7769999999999998e-06 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848754+0000", 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2869999999999999e-06 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877468+0000", 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028714185 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877541+0000", 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2927000000000002e-05 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877607+0000", 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5560000000000002e-05 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96969 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:28.396 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51733267900000002, 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028944845, 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96969 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848757+0000", 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848761+0000", 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848763+0000", 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.311e-06 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877630+0000", 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866326000000001 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877666+0000", 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6127000000000002e-05 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877701+0000", 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5413000000000001e-05 2024-09-13T07:13:28.400 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96970 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51732368799999995, 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029100663999999998, 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.401 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96970 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848766+0000", 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.097e-06 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848769+0000", 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848772+0000", 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4269999999999998e-06 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877729+0000", 2024-09-13T07:13:28.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028957488 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877800+0000", 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1113999999999997e-05 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877866+0000", 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5882000000000004e-05 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96971 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51731555799999995, 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029181689, 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96971 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848774+0000", 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1209999999999999e-06 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848777+0000", 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.926e-06 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848780+0000", 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877884+0000", 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029104076999999999 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877919+0000", 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5655999999999999e-05 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877955+0000", 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5782999999999999e-05 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96972 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51730736700000002, 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029338843999999999, 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.410 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96972 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:28.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848781+0000", 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848783+0000", 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1659999999999999e-06 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848785+0000", 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.818e-06 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848788+0000", 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4569999999999999e-06 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877984+0000", 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029196203 2024-09-13T07:13:28.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878055+0000", 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0622000000000002e-05 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878120+0000", 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5816999999999997e-05 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96973 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.517282197, 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029407188000000001, 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.415 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96973 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:28.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848806+0000", 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848808+0000", 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7039999999999999e-06 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848813+0000", 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6360000000000002e-06 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848815+0000", 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5799999999999999e-06 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878143+0000", 2024-09-13T07:13:28.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029327678999999999 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878179+0000", 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5587000000000003e-05 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878214+0000", 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5456000000000001e-05 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96974 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51727089800000003, 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029557028999999999, 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96974 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848821+0000", 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5140000000000002e-06 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848825+0000", 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848827+0000", 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.39e-06 2024-09-13T07:13:28.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878240+0000", 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029413174 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878310+0000", 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9813000000000004e-05 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878375+0000", 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5122000000000004e-05 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96975 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51725985699999999, 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029640264, 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96975 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:28.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848830+0000", 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.235e-06 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848833+0000", 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.796e-06 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848835+0000", 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.402e-06 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878397+0000", 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029561548999999999 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878434+0000", 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6721000000000003e-05 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878469+0000", 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5855999999999997e-05 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96976 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.428 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51725173700000004, 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029795095000000001, 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96976 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:28.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848838+0000", 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.156e-06 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848841+0000", 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9110000000000002e-06 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848843+0000", 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878496+0000", 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029652371 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878567+0000", 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0784000000000005e-05 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878632+0000", 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5783000000000004e-05 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96977 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51724360599999997, 2024-09-13T07:13:28.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029869717, 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96977 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848846+0000", 2024-09-13T07:13:28.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1000000000000001e-06 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848849+0000", 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.835e-06 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848851+0000", 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878643+0000", 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029791405 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878678+0000", 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5139999999999999e-05 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878715+0000", 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7119000000000002e-05 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.437 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96978 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.517235589, 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030029390999999999, 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.438 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96978 2024-09-13T07:13:28.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.440 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:28.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848854+0000", 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1039999999999999e-06 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848857+0000", 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8399999999999999e-06 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848859+0000", 2024-09-13T07:13:28.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878742+0000", 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029882140000000001 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878813+0000", 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1771000000000001e-05 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878883+0000", 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9457999999999998e-05 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96979 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51722765699999995, 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030106193999999999, 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.444 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96979 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848862+0000", 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.234e-06 2024-09-13T07:13:28.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848865+0000", 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.836e-06 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848868+0000", 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878895+0000", 2024-09-13T07:13:28.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030027642 2024-09-13T07:13:28.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878931+0000", 2024-09-13T07:13:28.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5624999999999999e-05 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878968+0000", 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6720000000000001e-05 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96980 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.517219498, 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030267181000000001, 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96980 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848869+0000", 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:28.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.088e-06 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848873+0000", 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9019999999999999e-06 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848875+0000", 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1950000000000002e-06 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878996+0000", 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03012068 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879067+0000", 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1100999999999996e-05 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879137+0000", 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9424999999999993e-05 2024-09-13T07:13:28.453 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96981 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51720567799999995, 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030344368, 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96981 2024-09-13T07:13:28.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848885+0000", 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848888+0000", 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:13:28.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848891+0000", 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.554e-06 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879159+0000", 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030268155000000001 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879192+0000", 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879228+0000", 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6146999999999998e-05 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96982 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 0.51719641299999997, 2024-09-13T07:13:28.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030488562, 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96982 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848892+0000", 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848894+0000", 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2079999999999999e-06 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848896+0000", 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8679999999999999e-06 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848899+0000", 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879246+0000", 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030346838000000001 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879317+0000", 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1254000000000006e-05 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879381+0000", 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4331000000000005e-05 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.462 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:28.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:13:28.464 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:28.465+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:28.465 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:28.565 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:28.566+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:28.566 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T07:13:28.610 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:28.610 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:28.610 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:28.611 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:28.620 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:28.666 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:28.666+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:28.723 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:28.723 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:28.731 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:28.775 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:28.776+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:28.811 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:28.822 INFO:teuthology.orchestra.run.smithi031.stdout:5442 write 0xb24ae7c thru 0xb24ef9b (0x4120 bytes) 2024-09-13T07:13:28.825 INFO:teuthology.orchestra.run.smithi031.stdout:5443 writesame 0x7cb4668 thru 0x7cb6ee3 data_size 0x2(0x287c bytes) 2024-09-13T07:13:28.829 INFO:teuthology.orchestra.run.smithi031.stdout:5444 writesame 0x68509a3 thru 0x6856a35 data_size 0x9(0x6093 bytes) 2024-09-13T07:13:28.834 INFO:teuthology.orchestra.run.smithi031.stdout:5446 punch from 0x46fe786 to 0x470318a, (0x4a04 bytes) 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 50.799897260000002, 2024-09-13T07:13:28.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:13:28.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:13:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.252988387999999, 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.843 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:13:28.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88519 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:28.848 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.2518305539999997, 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017910447999999999, 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88519 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585257+0000", 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585267+0000", 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0635000000000001e-05 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585289+0000", 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1846999999999999e-05 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585299+0000", 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4949999999999997e-06 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585347+0000", 2024-09-13T07:13:28.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7672e-05 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585500+0000", 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015363700000000001 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.588596+0000", 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030953669999999999 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603068+0000", 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014472063 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603106+0000", 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8435e-05 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603142+0000", 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6488e-05 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603169+0000", 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6959000000000001e-05 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:89021 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.3073536409999997, 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02102269, 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.854 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 89021 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:28.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529734+0000", 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529742+0000", 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8290000000000008e-06 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529757+0000", 2024-09-13T07:13:28.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4868999999999999e-05 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529763+0000", 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.93e-06 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529803+0000", 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0404999999999997e-05 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:28.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529809+0000", 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531015+0000", 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012062049999999999 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531057+0000", 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1921000000000001e-05 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.534526+0000", 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034694909999999999 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550683+0000", 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016157169999999998 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550697+0000", 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3351e-05 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550731+0000", 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3997e-05 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550759+0000", 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7861e-05 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95026 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.860 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1413395749999999, 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014972104999999999, 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95026 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:28.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695752+0000", 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.283e-06 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695758+0000", 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695762+0000", 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710581+0000", 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014818934000000001 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710653+0000", 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2457000000000001e-05 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710722+0000", 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8633999999999994e-05 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95027 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.141323466, 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015079646, 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95027 2024-09-13T07:13:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695768+0000", 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695773+0000", 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.592e-06 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695778+0000", 2024-09-13T07:13:28.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710749+0000", 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014970809 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710805+0000", 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6149000000000001e-05 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710846+0000", 2024-09-13T07:13:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0840999999999997e-05 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95028 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1413077700000001, 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015231216000000001, 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95028 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695781+0000", 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695783+0000", 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9250000000000002e-06 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695789+0000", 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.835e-06 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695793+0000", 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3000000000000003e-06 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710881+0000", 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015087695999999999 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710947+0000", 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6169999999999998e-05 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.711013+0000", 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5643999999999997e-05 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95080 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.139706643, 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014948011000000001, 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.873 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95080 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:28.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697385+0000", 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.187e-06 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697388+0000", 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9459999999999998e-06 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697390+0000", 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5600000000000001e-06 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712211+0000", 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014820224 2024-09-13T07:13:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712276+0000", 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5650999999999999e-05 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712331+0000", 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4727e-05 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95081 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1396971869999999, 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015124461000000001, 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95081 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697394+0000", 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.418e-06 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697397+0000", 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697399+0000", 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.508e-06 2024-09-13T07:13:28.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712367+0000", 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01496809 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712442+0000", 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5290999999999995e-05 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712517+0000", 2024-09-13T07:13:28.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4396999999999999e-05 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95082 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1396885029999999, 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015249593000000001, 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.882 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95082 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697402+0000", 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.311e-06 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697405+0000", 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697407+0000", 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712557+0000", 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015149207 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712609+0000", 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2286e-05 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712651+0000", 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1687999999999998e-05 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95083 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3727360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.139680013, 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015418032, 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95083 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697409+0000", 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.049e-06 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697414+0000", 2024-09-13T07:13:28.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.783e-06 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697417+0000", 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.587e-06 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712683+0000", 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015266083 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712759+0000", 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6627999999999999e-05 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712828+0000", 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8121999999999996e-05 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95084 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3776512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.890 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1396707420000001, 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015560297000000001, 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95084 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697420+0000", 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697423+0000", 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3359999999999999e-06 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697425+0000", 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.452e-06 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712864+0000", 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015438927999999999 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712938+0000", 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3458e-05 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712979+0000", 2024-09-13T07:13:28.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1251999999999997e-05 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95085 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1396599519999999, 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015726161999999998, 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.895 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95085 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697429+0000", 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697431+0000", 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5230000000000001e-06 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697434+0000", 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697436+0000", 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3889999999999999e-06 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713008+0000", 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015571295000000001 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713084+0000", 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6537000000000003e-05 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713156+0000", 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1616e-05 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95086 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3874816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.139650947, 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015894697999999999, 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95086 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697438+0000", 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697440+0000", 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.159e-06 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697443+0000", 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9359999999999999e-06 2024-09-13T07:13:28.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697445+0000", 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713191+0000", 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015746343999999999 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713269+0000", 2024-09-13T07:13:28.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7737000000000005e-05 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713333+0000", 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4233000000000007e-05 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95087 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1396426079999999, 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01604939, 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.903 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95087 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697448+0000", 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.094e-06 2024-09-13T07:13:28.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697451+0000", 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8949999999999998e-06 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697454+0000", 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.999e-06 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713361+0000", 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015907092000000001 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713433+0000", 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2606000000000002e-05 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713496+0000", 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2908000000000001e-05 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95088 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1396324680000001, 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016219289000000001, 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95088 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:28.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697459+0000", 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.764e-06 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697462+0000", 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4240000000000002e-06 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697464+0000", 2024-09-13T07:13:28.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3989999999999998e-06 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713532+0000", 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016067403000000001 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713607+0000", 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5476999999999996e-05 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713676+0000", 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9115999999999994e-05 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95089 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1396226149999999, 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016381144, 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95089 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:28.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697468+0000", 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.734e-06 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697474+0000", 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697479+0000", 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713703+0000", 2024-09-13T07:13:28.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016224643 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4828999999999998e-05 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713848+0000", 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0029000000000003e-05 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.916 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95090 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.139606817, 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016573104000000002, 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95090 2024-09-13T07:13:28.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697482+0000", 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697484+0000", 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.916e-06 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697496+0000", 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697506+0000", 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0477e-05 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713903+0000", 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016396758000000001 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713981+0000", 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7892999999999994e-05 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714056+0000", 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5214000000000002e-05 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.921 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95091 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.1395733960000001, 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016674768, 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95091 2024-09-13T07:13:28.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697518+0000", 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4789999999999999e-06 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697527+0000", 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3799999999999994e-06 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697532+0000", 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714080+0000", 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016547798999999998 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714135+0000", 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5294000000000003e-05 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714191+0000", 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5606999999999998e-05 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95092 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4169728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.139552608, 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017373829, 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:28.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95092 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:28.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697536+0000", 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697540+0000", 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4879999999999999e-06 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697548+0000", 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0139999999999998e-06 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697553+0000", 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2460000000000003e-06 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714754+0000", 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017200653 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714831+0000", 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7156000000000005e-05 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714911+0000", 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9740999999999994e-05 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:28.930 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:28.932 INFO:teuthology.orchestra.run.smithi031.stdout:5447 trunc from 0xb24ef9c to 0x237653d 2024-09-13T07:13:28.932 INFO:teuthology.orchestra.run.smithi031.stdout:5448 trunc from 0x237653d to 0xa000901 2024-09-13T07:13:28.932 INFO:teuthology.orchestra.run.smithi031.stdout:5449 write 0xbfd5f6b thru 0xbfd9595 (0x362b bytes) 2024-09-13T07:13:28.932 INFO:teuthology.orchestra.run.smithi031.stdout:5450 write 0x5b9a71c thru 0x5ba6fe7 (0xc8cc bytes) 2024-09-13T07:13:28.932 INFO:teuthology.orchestra.run.smithi031.stdout:5452 trunc from 0xbfd9596 to 0x19d1f74 2024-09-13T07:13:28.933 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:28.933 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:28.932+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:28.963 INFO:teuthology.orchestra.run.smithi031.stdout:5453 writesame 0xe28b2b3 thru 0xe2978e0 data_size 0x1(0xc62e bytes) 2024-09-13T07:13:28.963 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:28.963 INFO:teuthology.orchestra.run.smithi031.stdout:5456 punch from 0x83c5e37 to 0x83c7626, (0x17ef bytes) 2024-09-13T07:13:28.963 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:28.963 INFO:teuthology.orchestra.run.smithi031.stdout:5458 writesame 0xceb563d thru 0xceb604e data_size 0x1(0xa12 bytes) 2024-09-13T07:13:28.966 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:28.967 INFO:teuthology.orchestra.run.smithi031.stdout:5460 write 0xb711178 thru 0xb71d9b0 (0xc839 bytes) 2024-09-13T07:13:28.970 INFO:teuthology.orchestra.run.smithi031.stdout:5461 write 0x31fe05a thru 0x320945f (0xb406 bytes) 2024-09-13T07:13:28.971 INFO:teuthology.orchestra.run.smithi031.stdout:5462 read 0x8789352 thru 0x878b604 (0x22b3 bytes) 2024-09-13T07:13:28.986 INFO:teuthology.orchestra.run.smithi031.stdout:5463 trunc from 0xe2978e1 to 0x1300f89 2024-09-13T07:13:29.003 INFO:teuthology.orchestra.run.smithi031.stdout:5466 punch from 0x922b6a to 0x92ceb7, (0xa34d bytes) 2024-09-13T07:13:29.003 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:29.014 INFO:teuthology.orchestra.run.smithi031.stdout:5469 trunc from 0x1300f89 to 0xa9fc1b4 2024-09-13T07:13:29.019 INFO:teuthology.orchestra.run.smithi031.stdout:5470 writesame 0xc545e5 thru 0xc6028b data_size 0xd(0xbca7 bytes) 2024-09-13T07:13:29.024 INFO:teuthology.orchestra.run.smithi031.stdout:5471 trunc from 0xa9fc1b4 to 0xcc2f1fe 2024-09-13T07:13:29.029 INFO:teuthology.orchestra.run.smithi031.stdout:5472 write 0x6628140 thru 0x662a3fd (0x22be bytes) 2024-09-13T07:13:29.032 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:29.032 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:29.033+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:29.034 INFO:teuthology.orchestra.run.smithi031.stdout:5474 trunc from 0xcc2f1fe to 0xaeb5d60 2024-09-13T07:13:29.039 INFO:teuthology.orchestra.run.smithi031.stdout:5475 punch from 0x63e9b7e to 0x63ebaef, (0x1f71 bytes) 2024-09-13T07:13:29.039 INFO:teuthology.orchestra.run.smithi031.stdout:5476 write 0x793087d thru 0x79379b7 (0x713b bytes) 2024-09-13T07:13:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:5477 writesame 0xd40ff50 thru 0xd41c2d0 data_size 0x53(0xc381 bytes) 2024-09-13T07:13:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:5478 punch from 0x8bdc94b to 0x8be3cb3, (0x7368 bytes) 2024-09-13T07:13:29.051 INFO:teuthology.orchestra.run.smithi031.stdout:5479 write 0xa61da94 thru 0xa62c439 (0xe9a6 bytes) 2024-09-13T07:13:29.054 INFO:teuthology.orchestra.run.smithi031.stdout:5480 write 0x891b9c3 thru 0x892a2ef (0xe92d bytes) 2024-09-13T07:13:29.059 INFO:teuthology.orchestra.run.smithi031.stdout:5481 trunc from 0xd41c2d1 to 0xaf997f4 2024-09-13T07:13:29.066 INFO:teuthology.orchestra.run.smithi031.stdout:5482 clone 32 order 25 su 8388608 sc 8 2024-09-13T07:13:29.079 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:29.079 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:29.079 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:29.080 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:29.088 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:29.133 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:29.134+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:29.190 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:29.190 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:29.190 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:29.190 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:29.190 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:29.198 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:29.242 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:29.243+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:29.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:29.301 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.946995606, 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99762887700000002, 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:29.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:13:29.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351641+0000", 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99754909000000003 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351653+0000", 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2248e-05 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.306 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88604 3.e 3:75ae40da:::rbd_header.117390733e87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5781645129999999, 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20280552800000001, 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88604 2024-09-13T07:13:29.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722855+0000", 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722859+0000", 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722870+0000", 2024-09-13T07:13:29.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0434999999999999e-05 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722875+0000", 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722912+0000", 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6943000000000002e-05 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722963+0000", 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1103000000000002e-05 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723070+0000", 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106279 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723472+0000", 2024-09-13T07:13:29.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040248200000000001 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925590+0000", 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20211748900000001 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925624+0000", 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4801000000000001e-05 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925661+0000", 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6643999999999997e-05 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88605 3.e 3:75ae40da:::rbd_header.117390733e87:head [watch ping cookie 139824472318400 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.5279574460000003, 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15274379900000001, 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.313 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88605 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773062+0000", 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773066+0000", 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1570000000000002e-06 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773073+0000", 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5490000000000003e-06 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773080+0000", 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773125+0000", 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4849999999999999e-05 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for rw locks", 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773190+0000", 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5567000000000004e-05 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925689+0000", 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15249922899999999 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925746+0000", 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6369999999999997e-05 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925806+0000", 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0668999999999998e-05 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94703 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3366912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6158393960000001, 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027375494, 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.318 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94703 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685180+0000", 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685183+0000", 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:13:29.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685189+0000", 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0730000000000001e-06 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685193+0000", 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712474+0000", 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027280787000000001 2024-09-13T07:13:29.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712507+0000", 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2786000000000001e-05 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712556+0000", 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9203000000000003e-05 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94704 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6158220079999999, 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027564668000000001, 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94704 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.323 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685200+0000", 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685207+0000", 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685212+0000", 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1449999999999999e-06 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712597+0000", 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027385158 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712681+0000", 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3961000000000005e-05 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712763+0000", 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1558999999999996e-05 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94705 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.327 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615791983, 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027678611999999998, 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94705 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:29.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685230+0000", 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4660000000000002e-06 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685237+0000", 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685242+0000", 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8729999999999998e-06 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712793+0000", 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027551022000000001 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712835+0000", 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2537000000000003e-05 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712907+0000", 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1556000000000004e-05 2024-09-13T07:13:29.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94706 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615774442, 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027861107999999999, 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94706 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685245+0000", 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685247+0000", 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9479999999999998e-06 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685253+0000", 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685258+0000", 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.549e-06 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712946+0000", 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027688252 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713033+0000", 2024-09-13T07:13:29.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7472000000000005e-05 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713107+0000", 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3478000000000004e-05 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94707 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615758912, 2024-09-13T07:13:29.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028004273, 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94707 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685263+0000", 2024-09-13T07:13:29.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8449999999999999e-06 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685283+0000", 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0165000000000001e-05 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685289+0000", 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7189999999999998e-06 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713138+0000", 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027849056000000001 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713189+0000", 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1801000000000001e-05 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713265+0000", 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5983999999999996e-05 2024-09-13T07:13:29.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94708 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615726574, 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028161510000000001, 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94708 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:29.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685296+0000", 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-06 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685302+0000", 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685306+0000", 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713301+0000", 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027995176 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713372+0000", 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0870000000000004e-05 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713455+0000", 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2956999999999999e-05 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.345 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94709 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615709735, 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357679, 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94709 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685312+0000", 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685318+0000", 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685323+0000", 2024-09-13T07:13:29.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713505+0000", 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028182713000000002 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713592+0000", 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6397000000000006e-05 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713668+0000", 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6188999999999998e-05 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94710 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6156922790000001, 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028518801, 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94710 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685327+0000", 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685330+0000", 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3120000000000001e-06 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685337+0000", 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:13:29.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685342+0000", 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713700+0000", 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357838999999999 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:29.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8438999999999999e-05 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713847+0000", 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8540000000000004e-05 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94711 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6156736060000001, 2024-09-13T07:13:29.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640149, 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94711 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685348+0000", 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685355+0000", 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685359+0000", 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713875+0000", 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028516414 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713931+0000", 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713987+0000", 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6084e-05 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94712 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:29.358 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6156567179999999, 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028779301, 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94712 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685365+0000", 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685371+0000", 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7620000000000001e-06 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685375+0000", 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.42e-06 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714016+0000", 2024-09-13T07:13:29.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640591 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714079+0000", 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3288999999999997e-05 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714143+0000", 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3361999999999994e-05 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94713 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615640505, 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028911175000000001, 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94713 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.363 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685379+0000", 2024-09-13T07:13:29.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685381+0000", 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1509999999999998e-06 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685387+0000", 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685391+0000", 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5920000000000002e-06 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714169+0000", 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028777971999999999 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714235+0000", 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5782000000000001e-05 2024-09-13T07:13:29.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714291+0000", 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5525000000000001e-05 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94714 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6156002270000001, 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029006134999999999, 2024-09-13T07:13:29.367 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94714 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685419+0000", 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685430+0000", 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015e-05 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685441+0000", 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1049e-05 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685446+0000", 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714313+0000", 2024-09-13T07:13:29.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866640999999998 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714368+0000", 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5024e-05 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7843000000000002e-05 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94715 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.61556853, 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029140899000000001, 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94715 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685451+0000", 2024-09-13T07:13:29.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685454+0000", 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.627e-06 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685466+0000", 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2510000000000001e-05 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685472+0000", 2024-09-13T07:13:29.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714453+0000", 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028980599999999999 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714524+0000", 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1186999999999995e-05 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714592+0000", 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8668000000000001e-05 2024-09-13T07:13:29.375 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94716 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615543618, 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029223545, 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94716 2024-09-13T07:13:29.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685479+0000", 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4980000000000001e-06 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685485+0000", 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685489+0000", 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714617+0000", 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029128106000000001 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714662+0000", 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.456e-05 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714700+0000", 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8281000000000002e-05 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94717 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6155267129999999, 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029398906999999998, 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.380 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94717 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685495+0000", 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0499999999999999e-06 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685501+0000", 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9569999999999999e-06 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685506+0000", 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3249999999999998e-06 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714726+0000", 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029219218000000002 2024-09-13T07:13:29.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714804+0000", 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8610999999999997e-05 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714892+0000", 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8114000000000004e-05 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.384 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94718 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.615509434, 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029526177000000001, 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94718 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685510+0000", 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685513+0000", 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4360000000000001e-06 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685519+0000", 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685523+0000", 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714918+0000", 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029395260999999999 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714985+0000", 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6737999999999997e-05 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715037+0000", 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1712999999999998e-05 2024-09-13T07:13:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94719 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 1.6154929170000001, 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029657744, 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94719 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685529+0000", 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685535+0000", 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:13:29.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685539+0000", 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715062+0000", 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029522738 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715125+0000", 2024-09-13T07:13:29.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3229000000000001e-05 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715185+0000", 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9441999999999998e-05 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:29.393 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:29.395 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:29.395+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:29.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:29.407 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:29.433 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:29.433 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:29.434 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T07:13:29.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:13:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:13:29.464 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:29.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:29.495 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:29.496+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:29.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:29.516 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:29.517 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone31 from 0xaf997f4 (overlap 0x1300f89) to 0x0 2024-09-13T07:13:29.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:29.578 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T07:13:29.621 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:29.622+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:29.697 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:29.697+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:29.700 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:29.700 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:29.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:13:29.758 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 68, 2024-09-13T07:13:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "f667c631-bb9d-47bf-97c6-7217d8893244", 2024-09-13T07:13:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T07:12:08.346514+0000", 2024-09-13T07:13:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T07:13:29.443720+0000", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T07:13:13.343075+0000", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T07:12:10.626165+0000", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:13:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T07:13:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:19.268005+0000", 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 17, 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:29.766 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:20.513150+0000", 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:29.767 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:29.768 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:29.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:29.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T07:13:29.771 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:28.796583+0000", 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:13:29.773 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:13:29.774 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "68", 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 35, 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 68, 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:13:29.775 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:13:29.776 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:13:29.777 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:13:29.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c29fb342-d45b-48c1-9694-dd01e2a01325", 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:29.779 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 66, 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.780 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.781 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T07:13:29.782 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/1709347595", 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/1709347595", 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/1709347595", 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/1709347595", 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4b979b5c-a78b-461d-b795-f6f5c25cf652", 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.784 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.855316162109375, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 65, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T07:13:29.785 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.786 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.787 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.788 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3300827638", 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3300827638", 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3300827638", 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3300827638", 2024-09-13T07:13:29.789 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "11259f54-1134-495f-9281-d5132aff2780", 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:29.790 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:13:29.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3977233096", 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3977233096", 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3977233096", 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3977233096", 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:13:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "a11be2be-3803-4a23-bbf8-40ea110ccf41", 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/3290545828", 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/3290545828", 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/3290545828", 2024-09-13T07:13:29.800 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/3290545828", 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2b6941d9-29f5-4836-a61c-47ff27a7172e", 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:29.801 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6800", 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.802 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6801", 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6802", 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.803 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6803", 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6806", 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6807", 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6804", 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.805 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6805", 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6801/3543694628", 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6803/3543694628", 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6807/3543694628", 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6805/3543694628", 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.806 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "17e3a02d-28e3-4126-a8f2-c7783e524d62", 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 55, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6808", 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6810", 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.808 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6812", 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6814", 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6820", 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6821", 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.810 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6816", 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6818", 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6810/951246042", 2024-09-13T07:13:29.811 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6814/951246042", 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6821/951246042", 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6818/951246042", 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ddf29f57-7733-42fa-94ed-015a529fb689", 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.812 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6824", 2024-09-13T07:13:29.813 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6825", 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6826", 2024-09-13T07:13:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6827", 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6830", 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6831", 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6828", 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6829", 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6825/1941674146", 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6827/1941674146", 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6831/1941674146", 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6829/1941674146", 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d91e2eed-3bfb-4eb0-ac30-18770c106742", 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 13, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 30, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 53, 2024-09-13T07:13:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 31, 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6809", 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6811", 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6813", 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6815", 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6822", 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6823", 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.821 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6817", 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6819", 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6811/48890479", 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6815/48890479", 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6823/48890479", 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6819/48890479", 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:13:29.823 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.805494+0000", 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.824 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.817444+0000", 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.712083+0000", 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.739958+0000", 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.826 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.720767+0000", 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.715944+0000", 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.788853+0000", 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:13:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.809325+0000", 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 31 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.8", 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: 0, 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-13T07:13:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T07:13:29.830 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T07:13:29.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:29.832 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:29.832 INFO:teuthology.orchestra.run.smithi031.stdout:5483 trunc from 0xaf997f4 to 0x3378e7c 2024-09-13T07:13:29.834 INFO:tasks.thrashosds.thrasher:j is {'epoch': 68, 'fsid': 'f667c631-bb9d-47bf-97c6-7217d8893244', 'created': '2024-09-13T07:12:08.346514+0000', 'modified': '2024-09-13T07:13:29.443720+0000', 'last_up_change': '2024-09-13T07:13:13.343075+0000', 'last_in_change': '2024-09-13T07:12:10.626165+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:12:19.268005+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:12:20.513150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:12:28.796583+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 68, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'c29fb342-d45b-48c1-9694-dd01e2a01325', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 1709347595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 1709347595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 1709347595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 1709347595}]}, 'public_addr': '172.21.15.31:6809/1709347595', 'cluster_addr': '172.21.15.31:6811/1709347595', 'heartbeat_back_addr': '172.21.15.31:6815/1709347595', 'heartbeat_front_addr': '172.21.15.31:6813/1709347595', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4b979b5c-a78b-461d-b795-f6f5c25cf652', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.855316162109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3300827638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3300827638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3300827638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3300827638}]}, 'public_addr': '172.21.15.31:6825/3300827638', 'cluster_addr': '172.21.15.31:6827/3300827638', 'heartbeat_back_addr': '172.21.15.31:6831/3300827638', 'heartbeat_front_addr': '172.21.15.31:6829/3300827638', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '11259f54-1134-495f-9281-d5132aff2780', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3977233096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3977233096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3977233096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3977233096}]}, 'public_addr': '172.21.15.31:6801/3977233096', 'cluster_addr': '172.21.15.31:6803/3977233096', 'heartbeat_back_addr': '172.21.15.31:6807/3977233096', 'heartbeat_front_addr': '172.21.15.31:6805/3977233096', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a11be2be-3803-4a23-bbf8-40ea110ccf41', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 3290545828}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 3290545828}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 3290545828}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 3290545828}]}, 'public_addr': '172.21.15.31:6817/3290545828', 'cluster_addr': '172.21.15.31:6819/3290545828', 'heartbeat_back_addr': '172.21.15.31:6823/3290545828', 'heartbeat_front_addr': '172.21.15.31:6821/3290545828', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2b6941d9-29f5-4836-a61c-47ff27a7172e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 3543694628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 3543694628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 3543694628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 3543694628}]}, 'public_addr': '172.21.15.184:6801/3543694628', 'cluster_addr': '172.21.15.184:6803/3543694628', 'heartbeat_back_addr': '172.21.15.184:6807/3543694628', 'heartbeat_front_addr': '172.21.15.184:6805/3543694628', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '17e3a02d-28e3-4126-a8f2-c7783e524d62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6810', 'nonce': 951246042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6814', 'nonce': 951246042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 951246042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6818', 'nonce': 951246042}]}, 'public_addr': '172.21.15.184:6810/951246042', 'cluster_addr': '172.21.15.184:6814/951246042', 'heartbeat_back_addr': '172.21.15.184:6821/951246042', 'heartbeat_front_addr': '172.21.15.184:6818/951246042', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ddf29f57-7733-42fa-94ed-015a529fb689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 1941674146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 1941674146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 1941674146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 1941674146}]}, 'public_addr': '172.21.15.184:6825/1941674146', 'cluster_addr': '172.21.15.184:6827/1941674146', 'heartbeat_back_addr': '172.21.15.184:6831/1941674146', 'heartbeat_front_addr': '172.21.15.184:6829/1941674146', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd91e2eed-3bfb-4eb0-ac30-18770c106742', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 30, 'up_from': 53, 'up_thru': 56, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6809', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 48890479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6813', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 48890479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 48890479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6817', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 48890479}]}, 'public_addr': '172.21.15.184:6811/48890479', 'cluster_addr': '172.21.15.184:6815/48890479', 'heartbeat_back_addr': '172.21.15.184:6823/48890479', 'heartbeat_front_addr': '172.21.15.184:6819/48890479', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.805494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.817444+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.712083+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.739958+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.720767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.715944+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.788853+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.809325+0000', 'dead_epoch': 31}], 'pg_upmap': [{'pgid': '3.8', 'osds': [0, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:13:29.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:13:29.834 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:29.835 INFO:teuthology.orchestra.run.smithi031.stdout:5485 write 0xc1c5dd8 thru 0xc1cbd01 (0x5f2a bytes) 2024-09-13T07:13:29.835 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:29.834+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:29.835 INFO:teuthology.orchestra.run.smithi031.stdout:5486 trunc from 0xc1cbd02 to 0x749a952 2024-09-13T07:13:29.853 INFO:teuthology.orchestra.run.smithi031.stdout:5487 write 0xb9a853a thru 0xb9b2d19 (0xa7e0 bytes) 2024-09-13T07:13:29.856 INFO:teuthology.orchestra.run.smithi031.stdout:5490 read 0xb8b53cc thru 0xb8bab27 (0x575c bytes) 2024-09-13T07:13:29.865 INFO:teuthology.orchestra.run.smithi031.stdout:5491 writesame 0xd42fb1f thru 0xd43c590 data_size 0x1(0xca72 bytes) 2024-09-13T07:13:29.868 INFO:teuthology.orchestra.run.smithi031.stdout:5492 punch from 0xbf17fb0 to 0xbf19098, (0x10e8 bytes) 2024-09-13T07:13:29.873 INFO:teuthology.orchestra.run.smithi031.stdout:5493 trunc from 0xd43c591 to 0x7b67830 2024-09-13T07:13:29.880 INFO:teuthology.orchestra.run.smithi031.stdout:5494 write 0x35ae3e8 thru 0x35afaed (0x1706 bytes) 2024-09-13T07:13:29.881 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:29.881 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:29.882 INFO:teuthology.orchestra.run.smithi031.stdout:5495 write 0x6b3b4d2 thru 0x6b45be0 (0xa70f bytes) 2024-09-13T07:13:29.884 INFO:teuthology.orchestra.run.smithi031.stdout:5496 read 0x21e8246 thru 0x21e835f (0x11a bytes) 2024-09-13T07:13:29.888 INFO:teuthology.orchestra.run.smithi031.stdout:5497 trunc from 0x7b67830 to 0xb50567d 2024-09-13T07:13:29.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:29.901 INFO:teuthology.orchestra.run.smithi031.stdout:5498 writesame 0xd6bd80f thru 0xd6c26cb data_size 0x1(0x4ebd bytes) 2024-09-13T07:13:29.903 INFO:teuthology.orchestra.run.smithi031.stdout:5499 read 0x1f83548 thru 0x1f8dfb3 (0xaa6c bytes) 2024-09-13T07:13:29.903 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:29.908 INFO:teuthology.orchestra.run.smithi031.stdout:5501 trunc from 0xd6c26cc to 0x8610ac1 2024-09-13T07:13:29.917 INFO:teuthology.orchestra.run.smithi031.stdout:5502 punch from 0x66b7526 to 0x66c0e94, (0x996e bytes) 2024-09-13T07:13:29.917 INFO:teuthology.orchestra.run.smithi031.stdout:5503 punch from 0x606b196 to 0x6078a4a, (0xd8b4 bytes) 2024-09-13T07:13:29.929 INFO:teuthology.orchestra.run.smithi031.stdout:5505 writesame 0xd3b0179 thru 0xd3b5a80 data_size 0x58(0x5908 bytes) 2024-09-13T07:13:29.931 INFO:teuthology.orchestra.run.smithi031.stdout:5506 punch from 0xa929835 to 0xa93374b, (0x9f16 bytes) 2024-09-13T07:13:29.931 INFO:teuthology.orchestra.run.smithi031.stdout:5507 write 0xbcedd2a thru 0xbcf1a41 (0x3d18 bytes) 2024-09-13T07:13:29.932 INFO:teuthology.orchestra.run.smithi031.stdout:5508 write 0x94be37 thru 0x94dfb6 (0x2180 bytes) 2024-09-13T07:13:29.933 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:29.934+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:29.933 INFO:teuthology.orchestra.run.smithi031.stdout:5509 read 0x86bb3ae thru 0x86c3a06 (0x8659 bytes) 2024-09-13T07:13:29.934 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:29.934 INFO:teuthology.orchestra.run.smithi031.stdout:5512 writesame 0xe8451e thru 0xe8c927 data_size 0x1(0x840a bytes) 2024-09-13T07:13:29.950 INFO:teuthology.orchestra.run.smithi031.stdout:5513 trunc from 0xd3b5a81 to 0xb3020ce 2024-09-13T07:13:29.970 INFO:teuthology.orchestra.run.smithi031.stdout:5515 write 0xea99b6f thru 0xeaa878d (0xec1f bytes) 2024-09-13T07:13:29.972 INFO:teuthology.orchestra.run.smithi031.stdout:5516 punch from 0x79cc035 to 0x79d337f, (0x734a bytes) 2024-09-13T07:13:29.972 INFO:teuthology.orchestra.run.smithi031.stdout:5517 read 0x864c712 thru 0x8651070 (0x495f bytes) 2024-09-13T07:13:29.972 INFO:teuthology.orchestra.run.smithi031.stdout:5518 read 0xab4c342 thru 0xab4f8a7 (0x3566 bytes) 2024-09-13T07:13:29.973 INFO:teuthology.orchestra.run.smithi031.stdout:5519 write 0xa17fef0 thru 0xa18e21e (0xe32f bytes) 2024-09-13T07:13:29.975 INFO:teuthology.orchestra.run.smithi031.stdout:5520 write 0x336f8d7 thru 0x3377aa0 (0x81ca bytes) 2024-09-13T07:13:29.977 INFO:teuthology.orchestra.run.smithi031.stdout:5521 read 0x2244108 thru 0x2252b0b (0xea04 bytes) 2024-09-13T07:13:29.977 INFO:teuthology.orchestra.run.smithi031.stdout:5522 read 0x15dac48 thru 0x15e76f3 (0xcaac bytes) 2024-09-13T07:13:29.977 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:29.977 INFO:teuthology.orchestra.run.smithi031.stdout:5524 write 0x77ff9d2 thru 0x77ffd2c (0x35b bytes) 2024-09-13T07:13:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:5526 punch from 0xd5deeea to 0xd5ee013, (0xf129 bytes) 2024-09-13T07:13:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:5527 punch from 0x192f6df to 0x193d889, (0xe1aa bytes) 2024-09-13T07:13:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:5528 write 0x3e4adc5 thru 0x3e57009 (0xc245 bytes) 2024-09-13T07:13:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:5529 punch from 0x147bdf6 to 0x148444b, (0x8655 bytes) 2024-09-13T07:13:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:5530 read 0x5081ea6 thru 0x508de9f (0xbffa bytes) 2024-09-13T07:13:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:5531 flatten 2024-09-13T07:13:29.987 INFO:teuthology.orchestra.run.smithi031.stdout:5533 punch from 0xb51e572 to 0xb5201f1, (0x1c7f bytes) 2024-09-13T07:13:29.988 INFO:teuthology.orchestra.run.smithi031.stdout:5534 write 0x3725548 thru 0x372b3a8 (0x5e61 bytes) 2024-09-13T07:13:29.989 INFO:teuthology.orchestra.run.smithi031.stdout:5535 writesame 0xbed0217 thru 0xbed7c21 data_size 0x1(0x7a0b bytes) 2024-09-13T07:13:30.004 INFO:teuthology.orchestra.run.smithi031.stdout:5537 trunc from 0xeaa878e to 0xebf4740 2024-09-13T07:13:30.011 INFO:teuthology.orchestra.run.smithi031.stdout:5538 write 0x3d944ac thru 0x3da333e (0xee93 bytes) 2024-09-13T07:13:30.013 INFO:teuthology.orchestra.run.smithi031.stdout:5539 read 0x522b3ab thru 0x522dda5 (0x29fb bytes) 2024-09-13T07:13:30.013 INFO:teuthology.orchestra.run.smithi031.stdout:5540 read 0x2692549 thru 0x269375e (0x1216 bytes) 2024-09-13T07:13:30.013 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.013 INFO:teuthology.orchestra.run.smithi031.stdout:5544 write 0x21991e0 thru 0x21a17b7 (0x85d8 bytes) 2024-09-13T07:13:30.015 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.015 INFO:teuthology.orchestra.run.smithi031.stdout:5547 read 0xc9eac9e thru 0xc9f6f84 (0xc2e7 bytes) 2024-09-13T07:13:30.015 INFO:teuthology.orchestra.run.smithi031.stdout:5548 write 0x7ad1a40 thru 0x7adc085 (0xa646 bytes) 2024-09-13T07:13:30.017 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.017 INFO:teuthology.orchestra.run.smithi031.stdout:5553 write 0x8b83a82 thru 0x8b8543c (0x19bb bytes) 2024-09-13T07:13:30.018 INFO:teuthology.orchestra.run.smithi031.stdout:5554 write 0x7b967b5 thru 0x7ba3a1e (0xd26a bytes) 2024-09-13T07:13:30.020 INFO:teuthology.orchestra.run.smithi031.stdout:5555 read 0x378095d thru 0x3781387 (0xa2b bytes) 2024-09-13T07:13:30.020 INFO:teuthology.orchestra.run.smithi031.stdout:5556 writesame 0x9537a61 thru 0x95462e8 data_size 0x1c(0xe888 bytes) 2024-09-13T07:13:30.022 INFO:teuthology.orchestra.run.smithi031.stdout:5557 punch from 0xd410990 to 0xd418a7c, (0x80ec bytes) 2024-09-13T07:13:30.022 INFO:teuthology.orchestra.run.smithi031.stdout:5559 write 0xad79d27 thru 0xad89186 (0xf460 bytes) 2024-09-13T07:13:30.024 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.024 INFO:teuthology.orchestra.run.smithi031.stdout:5561 read 0x7d69777 thru 0x7d6b9e1 (0x226b bytes) 2024-09-13T07:13:30.024 INFO:teuthology.orchestra.run.smithi031.stdout:5563 writesame 0xc11e339 thru 0xc12c4bc data_size 0x66(0xe184 bytes) 2024-09-13T07:13:30.027 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.027 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.031 INFO:teuthology.orchestra.run.smithi031.stdout:5564 trunc from 0xebf4740 to 0xa235ae5 2024-09-13T07:13:30.033 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:30.034+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:30.039 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.039 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:30.041 INFO:teuthology.orchestra.run.smithi031.stdout:5567 write 0xc13988 thru 0xc1e8b8 (0xaf31 bytes) 2024-09-13T07:13:30.041 INFO:teuthology.orchestra.run.smithi031.stdout:5568 write 0x3ee2a1f thru 0x3eeccb5 (0xa297 bytes) 2024-09-13T07:13:30.043 INFO:teuthology.orchestra.run.smithi031.stdout:5569 punch from 0x32d9d46 to 0x32e812a, (0xe3e4 bytes) 2024-09-13T07:13:30.043 INFO:teuthology.orchestra.run.smithi031.stdout:5570 write 0x26e6d37 thru 0x26f0959 (0x9c23 bytes) 2024-09-13T07:13:30.044 INFO:teuthology.orchestra.run.smithi031.stdout:5571 read 0x63378d8 thru 0x6343525 (0xbc4e bytes) 2024-09-13T07:13:30.044 INFO:teuthology.orchestra.run.smithi031.stdout:5572 write 0xa0b77e thru 0xa15d76 (0xa5f9 bytes) 2024-09-13T07:13:30.046 INFO:teuthology.orchestra.run.smithi031.stdout:5573 read 0x226030f thru 0x22636ad (0x339f bytes) 2024-09-13T07:13:30.046 INFO:teuthology.orchestra.run.smithi031.stdout:5578 write 0x5a886ef thru 0x5a9469a (0xbfac bytes) 2024-09-13T07:13:30.048 INFO:teuthology.orchestra.run.smithi031.stdout:5579 read 0x5caa645 thru 0x5cb76ef (0xd0ab bytes) 2024-09-13T07:13:30.053 INFO:teuthology.orchestra.run.smithi031.stdout:5580 trunc from 0xa235ae5 to 0x6a82995 2024-09-13T07:13:30.072 INFO:teuthology.orchestra.run.smithi031.stdout:5581 trunc from 0x6a82995 to 0xe00de72 2024-09-13T07:13:30.076 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.076 INFO:teuthology.orchestra.run.smithi031.stdout:5584 write 0xbdc4210 thru 0xbdc691c (0x270d bytes) 2024-09-13T07:13:30.077 INFO:teuthology.orchestra.run.smithi031.stdout:5585 punch from 0x70cfdc1 to 0x70d8184, (0x83c3 bytes) 2024-09-13T07:13:30.077 INFO:teuthology.orchestra.run.smithi031.stdout:5586 read 0xdad6374 thru 0xdadbc2d (0x58ba bytes) 2024-09-13T07:13:30.077 INFO:teuthology.orchestra.run.smithi031.stdout:5587 punch from 0x1c47ba7 to 0x1c490c6, (0x151f bytes) 2024-09-13T07:13:30.077 INFO:teuthology.orchestra.run.smithi031.stdout:5588 write 0x3b87435 thru 0x3b8eb8f (0x775b bytes) 2024-09-13T07:13:30.078 INFO:teuthology.orchestra.run.smithi031.stdout:5589 read 0x3b4664 thru 0x3b6e4d (0x27ea bytes) 2024-09-13T07:13:30.079 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.079 INFO:teuthology.orchestra.run.smithi031.stdout:5592 read 0xb303487 thru 0xb3081d8 (0x4d52 bytes) 2024-09-13T07:13:30.079 INFO:teuthology.orchestra.run.smithi031.stdout:5595 write 0x2ce7338 thru 0x2cebb2b (0x47f4 bytes) 2024-09-13T07:13:30.081 INFO:teuthology.orchestra.run.smithi031.stdout:5596 punch from 0x391f6d7 to 0x39290da, (0x9a03 bytes) 2024-09-13T07:13:30.081 INFO:teuthology.orchestra.run.smithi031.stdout:5598 write 0x1819d3a thru 0x181cc71 (0x2f38 bytes) 2024-09-13T07:13:30.082 INFO:teuthology.orchestra.run.smithi031.stdout:5599 writesame 0x1b29ee6 thru 0x1b2c05f data_size 0x5(0x217a bytes) 2024-09-13T07:13:30.084 INFO:teuthology.orchestra.run.smithi031.stdout:5600 writesame 0xdb5f65d thru 0xdb60295 data_size 0x15(0xc39 bytes) 2024-09-13T07:13:30.085 INFO:teuthology.orchestra.run.smithi031.stdout:5601 read 0x3175400 thru 0x31788ee (0x34ef bytes) 2024-09-13T07:13:30.085 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.085 INFO:teuthology.orchestra.run.smithi031.stdout:5604 writesame 0x92c3729 thru 0x92ccefa data_size 0x1(0x97d2 bytes) 2024-09-13T07:13:30.102 INFO:teuthology.orchestra.run.smithi031.stdout:5605 read 0x3fe796f thru 0x3fea9f0 (0x3082 bytes) 2024-09-13T07:13:30.102 INFO:teuthology.orchestra.run.smithi031.stdout:5607 write 0x7000d45 thru 0x700d3ce (0xc68a bytes) 2024-09-13T07:13:30.104 INFO:teuthology.orchestra.run.smithi031.stdout:5608 write 0xd162ec2 thru 0xd163bec (0xd2b bytes) 2024-09-13T07:13:30.106 INFO:teuthology.orchestra.run.smithi031.stdout:5609 writesame 0xa5a6576 thru 0xa5b2126 data_size 0x1(0xbbb1 bytes) 2024-09-13T07:13:30.130 INFO:teuthology.orchestra.run.smithi031.stdout:5610 writesame 0x61e67be thru 0x61e7211 data_size 0x2(0xa54 bytes) 2024-09-13T07:13:30.132 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.134 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:30.135+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:30.143 INFO:teuthology.orchestra.run.smithi031.stdout:5612 trunc from 0xe00de72 to 0x431a55b 2024-09-13T07:13:30.150 INFO:teuthology.orchestra.run.smithi031.stdout:5613 read 0x34b71 thru 0x3c5ed (0x7a7d bytes) 2024-09-13T07:13:30.155 INFO:teuthology.orchestra.run.smithi031.stdout:5614 trunc from 0x431a55b to 0x410486 2024-09-13T07:13:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:13:30.159 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:13:30.167 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.167 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.175 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-13T07:13:29.451201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":872843795,"num_objects":1131,"num_object_clones":331,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1131,"num_whiteouts":380,"num_read":118704,"num_read_kb":495231,"num_write":15179,"num_write_kb":286269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":30321983,"num_keys_recovered":48,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11218,"ondisk_log_size":11218,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":357608,"kb_used_data":46332,"kb_used_omap":20,"kb_used_meta":311147,"kb_avail":754617112,"statfs":{"total":773094113280,"available":772727922688,"internally_reserved":0,"allocated":47443968,"data_stored":355254448,"data_compressed":9828543,"data_compressed_allocated":26824704,"data_compressed_original":339566832,"omap_allocated":21248,"internal_metadata":318614784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9289600,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":2848,"num_read_kb":5085,"num_write":575,"num_write_kb":2147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027758"},"pg_stats":[{"pgid":"3.1f","version":"61'205","reported_seq":2950,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.536257+0000","last_change":"2024-09-13T07:12:32.166874+0000","last_active":"2024-09-13T07:13:23.536257+0000","last_peered":"2024-09-13T07:13:23.536257+0000","last_clean":"2024-09-13T07:13:23.536257+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:13:23.536257+0000","last_undegraded":"2024-09-13T07:13:23.536257+0000","last_fullsized":"2024-09-13T07:13:23.536257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001055669,"stat_sum":{"num_bytes":27016969,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":2775,"num_read_kb":13803,"num_write":261,"num_write_kb":7996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"62'216","reported_seq":3298,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.644497+0000","last_change":"2024-09-13T07:12:32.154466+0000","last_active":"2024-09-13T07:13:23.644497+0000","last_peered":"2024-09-13T07:13:23.644497+0000","last_clean":"2024-09-13T07:13:23.644497+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:13:23.644497+0000","last_undegraded":"2024-09-13T07:13:23.644497+0000","last_fullsized":"2024-09-13T07:13:23.644497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058285399999999995,"stat_sum":{"num_bytes":27586141,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":3409,"num_read_kb":15894,"num_write":202,"num_write_kb":5462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"62'519","reported_seq":2259,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.543039+0000","last_change":"2024-09-13T07:12:32.162019+0000","last_active":"2024-09-13T07:13:23.543039+0000","last_peered":"2024-09-13T07:13:23.543039+0000","last_clean":"2024-09-13T07:13:23.543039+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:13:23.543039+0000","last_undegraded":"2024-09-13T07:13:23.543039+0000","last_fullsized":"2024-09-13T07:13:23.543039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093934900000000002,"stat_sum":{"num_bytes":28123595,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":4039,"num_read_kb":14396,"num_write":805,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'193","reported_seq":2436,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.424656+0000","last_change":"2024-09-13T07:12:32.161811+0000","last_active":"2024-09-13T07:13:24.424656+0000","last_peered":"2024-09-13T07:13:24.424656+0000","last_clean":"2024-09-13T07:13:24.424656+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:13:24.424656+0000","last_undegraded":"2024-09-13T07:13:24.424656+0000","last_fullsized":"2024-09-13T07:13:24.424656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082332300000000005,"stat_sum":{"num_bytes":33282355,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":2303,"num_read_kb":19791,"num_write":189,"num_write_kb":2203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"61'899","reported_seq":4211,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.416183+0000","last_change":"2024-09-13T07:12:32.166941+0000","last_active":"2024-09-13T07:13:23.416183+0000","last_peered":"2024-09-13T07:13:23.416183+0000","last_clean":"2024-09-13T07:13:23.416183+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:13:23.416183+0000","last_undegraded":"2024-09-13T07:13:23.416183+0000","last_fullsized":"2024-09-13T07:13:23.416183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001067222,"stat_sum":{"num_bytes":37703333,"num_objects":47,"num_object_clones":17,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":7538,"num_read_kb":33436,"num_write":1325,"num_write_kb":6762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"63'232","reported_seq":3027,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.468461+0000","last_change":"2024-09-13T07:12:32.157122+0000","last_active":"2024-09-13T07:13:24.468461+0000","last_peered":"2024-09-13T07:13:24.468461+0000","last_clean":"2024-09-13T07:13:24.468461+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:13:24.468461+0000","last_undegraded":"2024-09-13T07:13:24.468461+0000","last_fullsized":"2024-09-13T07:13:24.468461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074061500000000002,"stat_sum":{"num_bytes":21208327,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":11,"num_read":3283,"num_read_kb":18196,"num_write":245,"num_write_kb":6242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"62'259","reported_seq":4431,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.700936+0000","last_change":"2024-09-13T07:12:32.152866+0000","last_active":"2024-09-13T07:13:23.700936+0000","last_peered":"2024-09-13T07:13:23.700936+0000","last_clean":"2024-09-13T07:13:23.700936+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:13:23.700936+0000","last_undegraded":"2024-09-13T07:13:23.700936+0000","last_fullsized":"2024-09-13T07:13:23.700936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000412552,"stat_sum":{"num_bytes":28546107,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":4412,"num_read_kb":16878,"num_write":317,"num_write_kb":18390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"62'383","reported_seq":4403,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.197854+0000","last_change":"2024-09-13T07:12:32.154395+0000","last_active":"2024-09-13T07:13:24.197854+0000","last_peered":"2024-09-13T07:13:24.197854+0000","last_clean":"2024-09-13T07:13:24.197854+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:13:24.197854+0000","last_undegraded":"2024-09-13T07:13:24.197854+0000","last_fullsized":"2024-09-13T07:13:24.197854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054318499999999996,"stat_sum":{"num_bytes":31089595,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":6425,"num_read_kb":24386,"num_write":460,"num_write_kb":6962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"63'286","reported_seq":2420,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.421807+0000","last_change":"2024-09-13T07:12:32.167160+0000","last_active":"2024-09-13T07:13:24.421807+0000","last_peered":"2024-09-13T07:13:24.421807+0000","last_clean":"2024-09-13T07:13:24.421807+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:13:24.421807+0000","last_undegraded":"2024-09-13T07:13:24.421807+0000","last_fullsized":"2024-09-13T07:13:24.421807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013000990000000001,"stat_sum":{"num_bytes":26898736,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":12,"num_read":3212,"num_read_kb":15235,"num_write":277,"num_write_kb":6610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"61'186","reported_seq":1100,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.639154+0000","last_change":"2024-09-13T07:13:14.415606+0000","last_active":"2024-09-13T07:13:23.639154+0000","last_peered":"2024-09-13T07:13:23.639154+0000","last_clean":"2024-09-13T07:13:23.639154+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:13:23.639154+0000","last_undegraded":"2024-09-13T07:13:23.639154+0000","last_fullsized":"2024-09-13T07:13:23.639154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000795997,"stat_sum":{"num_bytes":24453565,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":821,"num_read_kb":7337,"num_write":165,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"62'240","reported_seq":4235,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T07:13:25.429641+0000","last_change":"2024-09-13T07:13:17.373195+0000","last_active":"2024-09-13T07:13:25.429641+0000","last_peered":"2024-09-13T07:13:25.429641+0000","last_clean":"2024-09-13T07:13:25.429641+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:13:25.429641+0000","last_undegraded":"2024-09-13T07:13:25.429641+0000","last_fullsized":"2024-09-13T07:13:25.429641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027228899999999999,"stat_sum":{"num_bytes":21497076,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":4085,"num_read_kb":14560,"num_write":339,"num_write_kb":7001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"61'307","reported_seq":1719,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.424860+0000","last_change":"2024-09-13T07:13:17.374397+0000","last_active":"2024-09-13T07:13:23.424860+0000","last_peered":"2024-09-13T07:13:23.424860+0000","last_clean":"2024-09-13T07:13:23.424860+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:13:23.424860+0000","last_undegraded":"2024-09-13T07:13:23.424860+0000","last_fullsized":"2024-09-13T07:13:23.424860+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035592099999999998,"stat_sum":{"num_bytes":40356097,"num_objects":37,"num_object_clones":13,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":2286,"num_read_kb":12566,"num_write":381,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":60,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T07:13:22.394286+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:13:22.394286+0000","last_peered":"2024-09-13T07:13:22.394286+0000","last_clean":"2024-09-13T07:13:22.394286+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:13:22.394286+0000","last_undegraded":"2024-09-13T07:13:22.394286+0000","last_fullsized":"2024-09-13T07:13:22.394286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"61'322","reported_seq":3391,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.639570+0000","last_change":"2024-09-13T07:12:32.162053+0000","last_active":"2024-09-13T07:13:23.639570+0000","last_peered":"2024-09-13T07:13:23.639570+0000","last_clean":"2024-09-13T07:13:23.639570+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:13:23.639570+0000","last_undegraded":"2024-09-13T07:13:23.639570+0000","last_fullsized":"2024-09-13T07:13:23.639570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091572500000000005,"stat_sum":{"num_bytes":13315710,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":4617,"num_read_kb":12610,"num_write":401,"num_write_kb":2232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T07:13:22.394330+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:13:22.394330+0000","last_peered":"2024-09-13T07:13:22.394330+0000","last_clean":"2024-09-13T07:13:22.394330+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:13:22.394330+0000","last_undegraded":"2024-09-13T07:13:22.394330+0000","last_fullsized":"2024-09-13T07:13:22.394330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"62'426","reported_seq":3449,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.303752+0000","last_change":"2024-09-13T07:12:32.153134+0000","last_active":"2024-09-13T07:13:24.303752+0000","last_peered":"2024-09-13T07:13:24.303752+0000","last_clean":"2024-09-13T07:13:24.303752+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:13:24.303752+0000","last_undegraded":"2024-09-13T07:13:24.303752+0000","last_fullsized":"2024-09-13T07:13:24.303752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047834,"stat_sum":{"num_bytes":22573513,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":5,"num_read":4549,"num_read_kb":9890,"num_write":577,"num_write_kb":6943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.424894+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:13:23.424894+0000","last_peered":"2024-09-13T07:13:23.424894+0000","last_clean":"2024-09-13T07:13:23.424894+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:13:23.424894+0000","last_undegraded":"2024-09-13T07:13:23.424894+0000","last_fullsized":"2024-09-13T07:13:23.424894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"62'291","reported_seq":668,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.300313+0000","last_change":"2024-09-13T07:13:17.372505+0000","last_active":"2024-09-13T07:13:24.300313+0000","last_peered":"2024-09-13T07:13:24.300313+0000","last_clean":"2024-09-13T07:13:24.300313+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:13:24.300313+0000","last_undegraded":"2024-09-13T07:13:24.300313+0000","last_fullsized":"2024-09-13T07:13:24.300313+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062209699999999995,"stat_sum":{"num_bytes":8819496,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":1084,"num_read_kb":4333,"num_write":409,"num_write_kb":1841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":61,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.419481+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:13:23.419481+0000","last_peered":"2024-09-13T07:13:23.419481+0000","last_clean":"2024-09-13T07:13:23.419481+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:13:23.419481+0000","last_undegraded":"2024-09-13T07:13:23.419481+0000","last_fullsized":"2024-09-13T07:13:23.419481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'243","reported_seq":6808,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.421428+0000","last_change":"2024-09-13T07:12:32.166255+0000","last_active":"2024-09-13T07:13:24.421428+0000","last_peered":"2024-09-13T07:13:24.421428+0000","last_clean":"2024-09-13T07:13:24.421428+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:13:24.421428+0000","last_undegraded":"2024-09-13T07:13:24.421428+0000","last_fullsized":"2024-09-13T07:13:24.421428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062381999999999997,"stat_sum":{"num_bytes":15562410,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":6752,"num_read_kb":17031,"num_write":306,"num_write_kb":16899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"62'403","reported_seq":2070,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.250364+0000","last_change":"2024-09-13T07:12:32.166064+0000","last_active":"2024-09-13T07:13:24.250364+0000","last_peered":"2024-09-13T07:13:24.250364+0000","last_clean":"2024-09-13T07:13:24.250364+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:13:24.250364+0000","last_undegraded":"2024-09-13T07:13:24.250364+0000","last_fullsized":"2024-09-13T07:13:24.250364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044591299999999998,"stat_sum":{"num_bytes":12541552,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":3019,"num_read_kb":14970,"num_write":565,"num_write_kb":6246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":182,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T07:13:25.429365+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:13:25.429365+0000","last_peered":"2024-09-13T07:13:25.429365+0000","last_clean":"2024-09-13T07:13:25.429365+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:13:25.429365+0000","last_undegraded":"2024-09-13T07:13:25.429365+0000","last_fullsized":"2024-09-13T07:13:25.429365+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":63,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.422951+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:13:23.422951+0000","last_peered":"2024-09-13T07:13:23.422951+0000","last_clean":"2024-09-13T07:13:23.422951+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:13:23.422951+0000","last_undegraded":"2024-09-13T07:13:23.422951+0000","last_fullsized":"2024-09-13T07:13:23.422951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"62'368","reported_seq":1831,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.385923+0000","last_change":"2024-09-13T07:12:32.152818+0000","last_active":"2024-09-13T07:13:24.385923+0000","last_peered":"2024-09-13T07:13:24.385923+0000","last_clean":"2024-09-13T07:13:24.385923+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:13:24.385923+0000","last_undegraded":"2024-09-13T07:13:24.385923+0000","last_fullsized":"2024-09-13T07:13:24.385923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038429300000000002,"stat_sum":{"num_bytes":38176098,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":2500,"num_read_kb":16602,"num_write":556,"num_write_kb":7113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T07:13:25.429459+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:13:25.429459+0000","last_peered":"2024-09-13T07:13:25.429459+0000","last_clean":"2024-09-13T07:13:25.429459+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:13:25.429459+0000","last_undegraded":"2024-09-13T07:13:25.429459+0000","last_fullsized":"2024-09-13T07:13:25.429459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"62'278","reported_seq":3350,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.083232+0000","last_change":"2024-09-13T07:12:32.156546+0000","last_active":"2024-09-13T07:13:24.083232+0000","last_peered":"2024-09-13T07:13:24.083232+0000","last_clean":"2024-09-13T07:13:24.083232+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:13:24.083232+0000","last_undegraded":"2024-09-13T07:13:24.083232+0000","last_fullsized":"2024-09-13T07:13:24.083232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000193666,"stat_sum":{"num_bytes":49253623,"num_objects":45,"num_object_clones":15,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":3549,"num_read_kb":18249,"num_write":423,"num_write_kb":23115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.413397+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:13:23.413397+0000","last_peered":"2024-09-13T07:13:23.413397+0000","last_clean":"2024-09-13T07:13:23.413397+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:13:23.413397+0000","last_undegraded":"2024-09-13T07:13:23.413397+0000","last_fullsized":"2024-09-13T07:13:23.413397+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"63'336","reported_seq":4288,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.426638+0000","last_change":"2024-09-13T07:12:32.154699+0000","last_active":"2024-09-13T07:13:24.426638+0000","last_peered":"2024-09-13T07:13:24.426638+0000","last_clean":"2024-09-13T07:13:24.426638+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:13:24.426638+0000","last_undegraded":"2024-09-13T07:13:24.426638+0000","last_fullsized":"2024-09-13T07:13:24.426638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065879899999999995,"stat_sum":{"num_bytes":49144005,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":5095,"num_read_kb":18087,"num_write":503,"num_write_kb":24221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T07:13:25.429358+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:13:25.429358+0000","last_peered":"2024-09-13T07:13:25.429358+0000","last_clean":"2024-09-13T07:13:25.429358+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:13:25.429358+0000","last_undegraded":"2024-09-13T07:13:25.429358+0000","last_fullsized":"2024-09-13T07:13:25.429358+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"unknown","last_fresh":"2024-09-13T07:13:24.421053+0000","last_change":"2024-09-13T07:13:24.501945+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T07:13:24.421053+0000","last_undegraded":"2024-09-13T07:13:24.421053+0000","last_fullsized":"2024-09-13T07:13:24.421053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:50:14.901075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"62'444","reported_seq":715,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.085777+0000","last_change":"2024-09-13T07:12:32.156602+0000","last_active":"2024-09-13T07:13:24.085777+0000","last_peered":"2024-09-13T07:13:24.085777+0000","last_clean":"2024-09-13T07:13:24.085777+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:13:24.085777+0000","last_undegraded":"2024-09-13T07:13:24.085777+0000","last_fullsized":"2024-09-13T07:13:24.085777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030558400000000002,"stat_sum":{"num_bytes":11115573,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":1715,"num_read_kb":4830,"num_write":716,"num_write_kb":9299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"62'612","reported_seq":2963,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.358249+0000","last_change":"2024-09-13T07:12:32.193641+0000","last_active":"2024-09-13T07:13:24.358249+0000","last_peered":"2024-09-13T07:13:24.358249+0000","last_clean":"2024-09-13T07:13:24.358249+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:13:24.358249+0000","last_undegraded":"2024-09-13T07:13:24.358249+0000","last_fullsized":"2024-09-13T07:13:24.358249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023357299999999999,"stat_sum":{"num_bytes":33248985,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":5836,"num_read_kb":16466,"num_write":811,"num_write_kb":5693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"62'450","reported_seq":3029,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.300127+0000","last_change":"2024-09-13T07:12:32.154134+0000","last_active":"2024-09-13T07:13:24.300127+0000","last_peered":"2024-09-13T07:13:24.300127+0000","last_clean":"2024-09-13T07:13:24.300127+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:13:24.300127+0000","last_undegraded":"2024-09-13T07:13:24.300127+0000","last_fullsized":"2024-09-13T07:13:24.300127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438474,"stat_sum":{"num_bytes":33331550,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":5805,"num_read_kb":15681,"num_write":495,"num_write_kb":8030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"62'190","reported_seq":1969,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:23.514422+0000","last_change":"2024-09-13T07:12:32.161253+0000","last_active":"2024-09-13T07:13:23.514422+0000","last_peered":"2024-09-13T07:13:23.514422+0000","last_clean":"2024-09-13T07:13:23.514422+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:13:23.514422+0000","last_undegraded":"2024-09-13T07:13:23.514422+0000","last_fullsized":"2024-09-13T07:13:23.514422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000322453,"stat_sum":{"num_bytes":29114993,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":2130,"num_read_kb":17303,"num_write":290,"num_write_kb":8520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"65'242","reported_seq":1702,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T07:13:27.113678+0000","last_change":"2024-09-13T07:13:17.375265+0000","last_active":"2024-09-13T07:13:27.113678+0000","last_peered":"2024-09-13T07:13:27.113678+0000","last_clean":"2024-09-13T07:13:27.113678+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:13:27.113678+0000","last_undegraded":"2024-09-13T07:13:27.113678+0000","last_fullsized":"2024-09-13T07:13:27.113678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034938500000000002,"stat_sum":{"num_bytes":14927746,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":1929,"num_read_kb":6127,"num_write":267,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"65'536","reported_seq":1535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T07:13:26.970572+0000","last_change":"2024-09-13T07:13:14.420801+0000","last_active":"2024-09-13T07:13:26.970572+0000","last_peered":"2024-09-13T07:13:26.970572+0000","last_clean":"2024-09-13T07:13:26.970572+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:13:26.970572+0000","last_undegraded":"2024-09-13T07:13:26.970572+0000","last_fullsized":"2024-09-13T07:13:26.970572+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050734399999999996,"stat_sum":{"num_bytes":24979816,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":7,"num_read":5614,"num_read_kb":22489,"num_write":714,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"65'886","reported_seq":6573,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T07:13:27.020268+0000","last_change":"2024-09-13T07:13:17.375323+0000","last_active":"2024-09-13T07:13:27.020268+0000","last_peered":"2024-09-13T07:13:27.020268+0000","last_clean":"2024-09-13T07:13:27.020268+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:13:27.020268+0000","last_undegraded":"2024-09-13T07:13:27.020268+0000","last_fullsized":"2024-09-13T07:13:27.020268+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052212899999999997,"stat_sum":{"num_bytes":34129467,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":9,"num_read":9409,"num_read_kb":13267,"num_write":1557,"num_write_kb":23631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"62'250","reported_seq":1110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.085753+0000","last_change":"2024-09-13T07:12:32.193735+0000","last_active":"2024-09-13T07:13:24.085753+0000","last_peered":"2024-09-13T07:13:24.085753+0000","last_clean":"2024-09-13T07:13:24.085753+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:13:24.085753+0000","last_undegraded":"2024-09-13T07:13:24.085753+0000","last_fullsized":"2024-09-13T07:13:24.085753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031914699999999999,"stat_sum":{"num_bytes":43041337,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":1281,"num_read_kb":17909,"num_write":315,"num_write_kb":20990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"65'239","reported_seq":3392,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T07:13:26.835087+0000","last_change":"2024-09-13T07:13:17.374529+0000","last_active":"2024-09-13T07:13:26.835087+0000","last_peered":"2024-09-13T07:13:26.835087+0000","last_clean":"2024-09-13T07:13:26.835087+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:13:26.835087+0000","last_undegraded":"2024-09-13T07:13:26.835087+0000","last_fullsized":"2024-09-13T07:13:26.835087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000333254,"stat_sum":{"num_bytes":32565241,"num_objects":44,"num_object_clones":15,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":3340,"num_read_kb":27029,"num_write":328,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"62'184","reported_seq":1565,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T07:13:24.085804+0000","last_change":"2024-09-13T07:13:17.374031+0000","last_active":"2024-09-13T07:13:24.085804+0000","last_peered":"2024-09-13T07:13:24.085804+0000","last_clean":"2024-09-13T07:13:24.085804+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:13:24.085804+0000","last_undegraded":"2024-09-13T07:13:24.085804+0000","last_fullsized":"2024-09-13T07:13:24.085804+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043838299999999998,"stat_sum":{"num_bytes":25872483,"num_objects":35,"num_object_clones":15,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":1260,"num_read_kb":12541,"num_write":188,"num_write_kb":4606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"65'473","reported_seq":3232,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T07:13:26.804430+0000","last_change":"2024-09-13T07:13:17.374820+0000","last_active":"2024-09-13T07:13:26.804430+0000","last_peered":"2024-09-13T07:13:26.804430+0000","last_clean":"2024-09-13T07:13:26.804430+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:13:26.804430+0000","last_undegraded":"2024-09-13T07:13:26.804430+0000","last_fullsized":"2024-09-13T07:13:26.804430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393278,"stat_sum":{"num_bytes":32777914,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":8,"num_read":4516,"num_read_kb":23239,"num_write":631,"num_write_kb":10961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":872253408,"num_objects":1127,"num_object_clones":331,"num_object_copies":2254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1127,"num_whiteouts":380,"num_read":118588,"num_read_kb":495131,"num_write":15018,"num_write_kb":283851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":30321983,"num_keys_recovered":48,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42020864,"data_stored":352973479,"data_compressed":9849856,"data_compressed_allocated":26877952,"data_compressed_original":339624176,"omap_allocated":7126,"internal_metadata":0},"log_size":11098,"ondisk_log_size":11098,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266692,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39416,"kb_used_data":6636,"kb_used_omap":9,"kb_used_meta":32758,"kb_avail":94332424,"statfs":{"total":96636764160,"available":96596402176,"internally_reserved":0,"allocated":6795264,"data_stored":50858134,"data_compressed":1312220,"data_compressed_allocated":4067328,"data_compressed_original":48884295,"omap_allocated":10093,"internal_metadata":33544339},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34348,"kb_used_data":3228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337492,"statfs":{"total":96636764160,"available":96601591808,"internally_reserved":0,"allocated":3305472,"data_stored":25242995,"data_compressed":754444,"data_compressed_allocated":1970176,"data_compressed_original":24370622,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.432}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.34100000000000003}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.377}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":5292,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":5419008,"data_stored":33742189,"data_compressed":864026,"data_compressed_allocated":2723840,"data_compressed_original":31648933,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38900000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.371},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.40699999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.317},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.307}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76644,"kb_used_data":9436,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94295196,"statfs":{"total":96636764160,"available":96558280704,"internally_reserved":0,"allocated":9662464,"data_stored":79330565,"data_compressed":1997340,"data_compressed_allocated":5804032,"data_compressed_original":76284473,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.38}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.28999999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.309}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45232,"kb_used_data":5928,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326608,"statfs":{"total":96636764160,"available":96590446592,"internally_reserved":0,"allocated":6070272,"data_stored":47925505,"data_compressed":1514984,"data_compressed_allocated":3260416,"data_compressed_original":45815302,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.311}]},{"osd":1,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.37}]},{"osd":4,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.42799999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33568,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338272,"statfs":{"total":96636764160,"available":96602390528,"internally_reserved":0,"allocated":2510848,"data_stored":11843560,"data_compressed":218232,"data_compressed_allocated":1060864,"data_compressed_original":10884498,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.318},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.30099999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.375},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.435},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.39700000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39740,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332100,"statfs":{"total":96636764160,"available":96596070400,"internally_reserved":0,"allocated":4636672,"data_stored":29684957,"data_compressed":885590,"data_compressed_allocated":2318336,"data_compressed_original":27946988,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.218},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.20000000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.27300000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.252},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.318}]},{"osd":4,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.35099999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.38}]},{"osd":6,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.308},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48152,"kb_used_data":8832,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94323688,"statfs":{"total":96636764160,"available":96587456512,"internally_reserved":0,"allocated":9043968,"data_stored":76626543,"data_compressed":2281707,"data_compressed_allocated":5619712,"data_compressed_original":73731721,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.3090000000000002,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":4.54,"5min":4.54,"15min":4.54},"last":0.315}]},{"osd":2,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34899999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.434},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.38200000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8331264,"data_stored":76358175,"data_compressed":2281966,"data_compressed_allocated":5623808,"data_compressed_original":73789065,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3903488,"data_stored":28808435,"data_compressed":877070,"data_compressed_allocated":2281472,"data_compressed_original":27373548,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":11640448,"data_compressed":218232,"data_compressed_allocated":1060864,"data_compressed_original":10884498,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5337088,"data_stored":47578907,"data_compressed":1514885,"data_compressed_allocated":3256320,"data_compressed_original":45798918,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9027584,"data_stored":79086547,"data_compressed":1997435,"data_compressed_allocated":5808128,"data_compressed_original":76292665,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4767744,"data_stored":33528026,"data_compressed":864285,"data_compressed_allocated":2727936,"data_compressed_original":31706277,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2584576,"data_stored":24914725,"data_compressed":754444,"data_compressed_allocated":1970176,"data_compressed_original":24370622,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6144000,"data_stored":51058216,"data_compressed":1341539,"data_compressed_allocated":4149248,"data_compressed_original":49408583,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:13:30.178 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [6, 0, 3, 2] 2024-09-13T07:13:30.178 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.4', '6', '0', '3', '2'] 2024-09-13T07:13:30.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.4 6 0 3 2 2024-09-13T07:13:30.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:30.180 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.191 INFO:teuthology.orchestra.run.smithi031.stdout:5617 write 0x4640f3c thru 0x464ec86 (0xdd4b bytes) 2024-09-13T07:13:30.201 INFO:teuthology.orchestra.run.smithi031.stdout:5618 trunc from 0x464ec87 to 0xb363531 2024-09-13T07:13:30.206 INFO:teuthology.orchestra.run.smithi031.stdout:5620 read 0x96040c thru 0x960b3e (0x733 bytes) 2024-09-13T07:13:30.206 INFO:teuthology.orchestra.run.smithi031.stdout:5622 writesame 0x3f73d79 thru 0x3f80c88 data_size 0x8(0xcf10 bytes) 2024-09-13T07:13:30.212 INFO:teuthology.orchestra.run.smithi031.stdout:5623 read 0x157c640 thru 0x1588ca5 (0xc666 bytes) 2024-09-13T07:13:30.217 INFO:teuthology.orchestra.run.smithi031.stdout:5624 trunc from 0xb363531 to 0x6b13be7 2024-09-13T07:13:30.224 INFO:teuthology.orchestra.run.smithi031.stdout:5625 writesame 0x6016e6e thru 0x6022e22 data_size 0x15(0xbfb5 bytes) 2024-09-13T07:13:30.233 INFO:teuthology.orchestra.run.smithi031.stdout:5626 trunc from 0x6b13be7 to 0xb7898fc 2024-09-13T07:13:30.235 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:30.236+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:30.238 INFO:teuthology.orchestra.run.smithi031.stdout:5627 read 0x777722c thru 0x77815c9 (0xa39e bytes) 2024-09-13T07:13:30.238 INFO:teuthology.orchestra.run.smithi031.stdout:5630 read 0x78690d8 thru 0x78745f3 (0xb51c bytes) 2024-09-13T07:13:30.239 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.248 INFO:teuthology.orchestra.run.smithi031.stdout:5633 trunc from 0xb7898fc to 0x3a63d07 2024-09-13T07:13:30.255 INFO:teuthology.orchestra.run.smithi031.stdout:5634 read 0x88a0eb thru 0x8978ea (0xd800 bytes) 2024-09-13T07:13:30.255 INFO:teuthology.orchestra.run.smithi031.stdout:5636 read 0xa6fb38 thru 0xa77bed (0x80b6 bytes) 2024-09-13T07:13:30.267 INFO:teuthology.orchestra.run.smithi031.stdout:5637 writesame 0x965cf3f thru 0x9667d22 data_size 0x3e(0xade4 bytes) 2024-09-13T07:13:30.268 INFO:teuthology.orchestra.run.smithi031.stdout:5639 read 0x5870a3f thru 0x5873746 (0x2d08 bytes) 2024-09-13T07:13:30.268 INFO:teuthology.orchestra.run.smithi031.stdout:5640 read 0x25f9ce6 thru 0x26064bf (0xc7da bytes) 2024-09-13T07:13:30.268 INFO:teuthology.orchestra.run.smithi031.stdout:5641 write 0x69ca15b thru 0x69ca1f2 (0x98 bytes) 2024-09-13T07:13:30.276 INFO:teuthology.orchestra.run.smithi031.stdout:5643 trunc from 0x9667d23 to 0x4b6da37 2024-09-13T07:13:30.292 INFO:teuthology.orchestra.run.smithi031.stdout:5645 write 0x6c3ea50 thru 0x6c45ac5 (0x7076 bytes) 2024-09-13T07:13:30.293 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.293 INFO:teuthology.orchestra.run.smithi031.stdout:5647 punch from 0x2de6c7b to 0x2df5d75, (0xf0fa bytes) 2024-09-13T07:13:30.293 INFO:teuthology.orchestra.run.smithi031.stdout:5648 read 0x26fe4f8 thru 0x270ca43 (0xe54c bytes) 2024-09-13T07:13:30.293 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.302 INFO:teuthology.orchestra.run.smithi031.stdout:5650 write 0x951f960 thru 0x952d6e8 (0xdd89 bytes) 2024-09-13T07:13:30.305 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.305 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.317 INFO:teuthology.orchestra.run.smithi031.stdout:5651 write 0xebe1fa1 thru 0xebe7d74 (0x5dd4 bytes) 2024-09-13T07:13:30.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:30.320 INFO:teuthology.orchestra.run.smithi031.stdout:5652 writesame 0x39ab99b thru 0x39b454a data_size 0x4(0x8bb0 bytes) 2024-09-13T07:13:30.327 INFO:teuthology.orchestra.run.smithi031.stdout:5655 trunc from 0xebe7d75 to 0xb51555e 2024-09-13T07:13:30.334 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:30.336 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:30.337+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:30.337 INFO:teuthology.orchestra.run.smithi031.stdout:5659 trunc from 0xb51555e to 0x8457200 2024-09-13T07:13:30.344 INFO:teuthology.orchestra.run.smithi031.stdout:5660 writesame 0x7a74b3 thru 0x7ab2a5 data_size 0x1(0x3df3 bytes) 2024-09-13T07:13:30.353 INFO:teuthology.orchestra.run.smithi031.stdout:5662 write 0x3ce4974 thru 0x3cf25bc (0xdc49 bytes) 2024-09-13T07:13:30.356 INFO:teuthology.orchestra.run.smithi031.stdout:5663 trunc from 0x8457200 to 0x719f51d 2024-09-13T07:13:30.363 INFO:teuthology.orchestra.run.smithi031.stdout:5664 read 0x1904cb7 thru 0x1906a42 (0x1d8c bytes) 2024-09-13T07:13:30.363 INFO:teuthology.orchestra.run.smithi031.stdout:5665 clone 33 order 18 su 131072 sc 7 2024-09-13T07:13:30.437 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:30.437+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:30.438 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.438 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:30.538 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:30.538+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:30.600 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T07:13:30.601 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.601 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.601 INFO:teuthology.orchestra.run.smithi031.stdout:5666 writesame 0x4328f22 thru 0x432a733 data_size 0x1(0x1812 bytes) 2024-09-13T07:13:30.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:30.611 INFO:teuthology.orchestra.run.smithi031.stdout:5668 read 0x1627b9e thru 0x162fa16 (0x7e79 bytes) 2024-09-13T07:13:30.611 INFO:teuthology.orchestra.run.smithi031.stdout:5670 write 0x25ad850 thru 0x25bc989 (0xf13a bytes) 2024-09-13T07:13:30.615 INFO:teuthology.orchestra.run.smithi031.stdout:5671 read 0x3688527 thru 0x369045e (0x7f38 bytes) 2024-09-13T07:13:30.616 INFO:teuthology.orchestra.run.smithi031.stdout:5672 clone 34 order 20 su 4096 sc 7 2024-09-13T07:13:30.639 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:30.640+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:30.717 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.717 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:30.740 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:30.741+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:30.841 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:30.842+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:30.873 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.873 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:30.942 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:30.943+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:30.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:31.080 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:31.081+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:31.143 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:31.144 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:31.144 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:31.144+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:31.152 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:31.245 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:31.246+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:31.261 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:31.261 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:31.270 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:31.346 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:31.346+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:31.412 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:31.412 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:31.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:31.464 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:31.465+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.4 pg_upmap_items mapping to [6->0,3->2] 2024-09-13T07:13:31.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:13:31.525 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone33 from 0x719f51d (overlap 0x719f51d) to 0x16b976c 2024-09-13T07:13:31.533 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:31.533 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:31.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:31.546 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:31.583 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:31.584+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:31.648 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:31.649+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:31.727 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:31.727 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:31.728 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:31.729 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:31.740 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:31.750 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:31.750+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:31.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:31.758 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:31.783 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:31.783 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:31.802 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:31.803 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:31.821 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:31.821 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T07:13:31.849 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:31.850+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:31.856 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:31.886 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:31.887 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:31.887 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:31.887 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:31.887 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:31.895 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:31.980 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:31.981+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:31.991 INFO:teuthology.orchestra.run.smithi031.stdout:5673 trunc from 0x719f51d to 0xd756bf 2024-09-13T07:13:32.045 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:32.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 30.381593382999998, 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.046 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.047 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:32.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 30.381555731999999, 2024-09-13T07:13:32.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.054 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 30.381536505, 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.055 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.056 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:32.057 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:32.058 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.505463092999999, 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.059 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:32.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:32.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96967 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1965257710000001, 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028701085000000001, 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96967 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848739+0000", 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848741+0000", 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.153e-06 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848743+0000", 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.723e-06 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848746+0000", 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.858e-06 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877369+0000", 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028623282999999999 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877406+0000", 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6139000000000001e-05 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877441+0000", 2024-09-13T07:13:32.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151000000000003e-05 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96968 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1965172099999997, 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028858604999999999, 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.069 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96968 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848749+0000", 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.091e-06 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848752+0000", 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7769999999999998e-06 2024-09-13T07:13:32.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848754+0000", 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2869999999999999e-06 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877468+0000", 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028714185 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877541+0000", 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2927000000000002e-05 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877607+0000", 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5560000000000002e-05 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96969 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.073 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1965090619999996, 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028944845, 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96969 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:32.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848757+0000", 2024-09-13T07:13:32.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848761+0000", 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848763+0000", 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.311e-06 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877630+0000", 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866326000000001 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877666+0000", 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6127000000000002e-05 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877701+0000", 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5413000000000001e-05 2024-09-13T07:13:32.077 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96970 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.196500071, 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029100663999999998, 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.078 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96970 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848766+0000", 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.097e-06 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848769+0000", 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848772+0000", 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4269999999999998e-06 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877729+0000", 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028957488 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877800+0000", 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1113999999999997e-05 2024-09-13T07:13:32.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877866+0000", 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5882000000000004e-05 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96971 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964919409999997, 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029181689, 2024-09-13T07:13:32.082 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96971 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848774+0000", 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1209999999999999e-06 2024-09-13T07:13:32.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848777+0000", 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.926e-06 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848780+0000", 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877884+0000", 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029104076999999999 2024-09-13T07:13:32.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877919+0000", 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5655999999999999e-05 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877955+0000", 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5782999999999999e-05 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96972 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964837499999996, 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029338843999999999, 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96972 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.087 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848781+0000", 2024-09-13T07:13:32.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848783+0000", 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1659999999999999e-06 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848785+0000", 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.818e-06 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848788+0000", 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4569999999999999e-06 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877984+0000", 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029196203 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878055+0000", 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0622000000000002e-05 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878120+0000", 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5816999999999997e-05 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96973 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964585799999998, 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029407188000000001, 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.091 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96973 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.092 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848806+0000", 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848808+0000", 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7039999999999999e-06 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848813+0000", 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6360000000000002e-06 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848815+0000", 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5799999999999999e-06 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878143+0000", 2024-09-13T07:13:32.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029327678999999999 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878179+0000", 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5587000000000003e-05 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878214+0000", 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5456000000000001e-05 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96974 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964472810000002, 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029557028999999999, 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96974 2024-09-13T07:13:32.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848821+0000", 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5140000000000002e-06 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848825+0000", 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848827+0000", 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.39e-06 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878240+0000", 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029413174 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878310+0000", 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9813000000000004e-05 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878375+0000", 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5122000000000004e-05 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96975 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964362399999997, 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029640264, 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.100 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96975 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.101 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848830+0000", 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.235e-06 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848833+0000", 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.796e-06 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848835+0000", 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.402e-06 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878397+0000", 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029561548999999999 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878434+0000", 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6721000000000003e-05 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878469+0000", 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5855999999999997e-05 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.104 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96976 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964281200000002, 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029795095000000001, 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96976 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848838+0000", 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.156e-06 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848841+0000", 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9110000000000002e-06 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848843+0000", 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878496+0000", 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029652371 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878567+0000", 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0784000000000005e-05 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878632+0000", 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5783000000000004e-05 2024-09-13T07:13:32.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96977 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964199889999998, 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029869717, 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.109 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96977 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848846+0000", 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1000000000000001e-06 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848849+0000", 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.835e-06 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848851+0000", 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878643+0000", 2024-09-13T07:13:32.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029791405 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878678+0000", 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5139999999999999e-05 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878715+0000", 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7119000000000002e-05 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.113 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96978 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1964119719999999, 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030029390999999999, 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96978 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848854+0000", 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1039999999999999e-06 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848857+0000", 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8399999999999999e-06 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848859+0000", 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878742+0000", 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029882140000000001 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878813+0000", 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1771000000000001e-05 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878883+0000", 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9457999999999998e-05 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96979 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.19640404, 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030106193999999999, 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.118 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96979 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848862+0000", 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.234e-06 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848865+0000", 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.836e-06 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848868+0000", 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878895+0000", 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030027642 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.121 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878931+0000", 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5624999999999999e-05 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878968+0000", 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6720000000000001e-05 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96980 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:32.122 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1963958809999999, 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030267181000000001, 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96980 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848869+0000", 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.088e-06 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848873+0000", 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9019999999999999e-06 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848875+0000", 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1950000000000002e-06 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878996+0000", 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03012068 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879067+0000", 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1100999999999996e-05 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879137+0000", 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9424999999999993e-05 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96981 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1963820609999996, 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030344368, 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96981 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848885+0000", 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848888+0000", 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848891+0000", 2024-09-13T07:13:32.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.554e-06 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879159+0000", 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030268155000000001 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879192+0000", 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879228+0000", 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6146999999999998e-05 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96982 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:32.131 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.1963727960000003, 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030488562, 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96982 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848892+0000", 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848894+0000", 2024-09-13T07:13:32.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2079999999999999e-06 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848896+0000", 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8679999999999999e-06 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848899+0000", 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879246+0000", 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030346838000000001 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879317+0000", 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1254000000000006e-05 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879381+0000", 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4331000000000005e-05 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.135 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.136 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.136 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:32.138 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:32.138 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:32.138+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5674 write 0xce67fa7 thru 0xce6b27b (0x32d5 bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5675 writesame 0xdde1018 thru 0xddec2f1 data_size 0x27(0xb2da bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5676 read 0x8799314 thru 0x87a2b71 (0x985e bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5677 read 0xd1663e0 thru 0xd16de23 (0x7a44 bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5678 punch from 0x6197abc to 0x619e409, (0x694d bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5681 read 0x7e5cece thru 0x7e5fac6 (0x2bf9 bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5683 read 0x5965c92 thru 0x596f291 (0x9600 bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5684 writesame 0xa178e49 thru 0xa188819 data_size 0x1(0xf9d1 bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:5685 write 0xe1e6b03 thru 0xe1e96e8 (0x2be6 bytes) 2024-09-13T07:13:32.139 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.140 INFO:teuthology.orchestra.run.smithi031.stdout:5688 write 0x3a67904 thru 0x3a7695d (0xf05a bytes) 2024-09-13T07:13:32.143 INFO:teuthology.orchestra.run.smithi031.stdout:5689 read 0x7d75a09 thru 0x7d83e5c (0xe454 bytes) 2024-09-13T07:13:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:5690 read 0xc7bf299 thru 0xc7cdd9d (0xeb05 bytes) 2024-09-13T07:13:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:5692 write 0xe4dd6e5 thru 0xe4e114c (0x3a68 bytes) 2024-09-13T07:13:32.165 INFO:teuthology.orchestra.run.smithi031.stdout:5694 trunc from 0xe4e114d to 0x5db91fa 2024-09-13T07:13:32.193 INFO:teuthology.orchestra.run.smithi031.stdout:5695 write 0xd329af8 thru 0xd32dbbc (0x40c5 bytes) 2024-09-13T07:13:32.203 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.203 INFO:teuthology.orchestra.run.smithi031.stdout:5697 punch from 0x990f2b5 to 0x991ced8, (0xdc23 bytes) 2024-09-13T07:13:32.203 INFO:teuthology.orchestra.run.smithi031.stdout:5698 punch from 0x9093349 to 0x909b0dc, (0x7d93 bytes) 2024-09-13T07:13:32.203 INFO:teuthology.orchestra.run.smithi031.stdout:5699 read 0x2e84379 thru 0x2e86dc7 (0x2a4f bytes) 2024-09-13T07:13:32.203 INFO:teuthology.orchestra.run.smithi031.stdout:5700 write 0xaaae1d0 thru 0xaab3788 (0x55b9 bytes) 2024-09-13T07:13:32.214 INFO:teuthology.orchestra.run.smithi031.stdout:5701 read 0x3de05af thru 0x3de6ae6 (0x6538 bytes) 2024-09-13T07:13:32.214 INFO:teuthology.orchestra.run.smithi031.stdout:5702 read 0x951e769 thru 0x9528cd2 (0xa56a bytes) 2024-09-13T07:13:32.215 INFO:teuthology.orchestra.run.smithi031.stdout:5703 read 0x50c375e thru 0x50d1b2c (0xe3cf bytes) 2024-09-13T07:13:32.215 INFO:teuthology.orchestra.run.smithi031.stdout:5704 punch from 0xcebbd35 to 0xcecb2b2, (0xf57d bytes) 2024-09-13T07:13:32.215 INFO:teuthology.orchestra.run.smithi031.stdout:5705 write 0x2a32f32 thru 0x2a345b5 (0x1684 bytes) 2024-09-13T07:13:32.224 INFO:teuthology.orchestra.run.smithi031.stdout:5706 read 0x792fa60 thru 0x793d236 (0xd7d7 bytes) 2024-09-13T07:13:32.224 INFO:teuthology.orchestra.run.smithi031.stdout:5707 punch from 0xa21ba7 to 0xa261fc, (0x4655 bytes) 2024-09-13T07:13:32.224 INFO:teuthology.orchestra.run.smithi031.stdout:5709 punch from 0x9d6f87e to 0x9d773f3, (0x7b75 bytes) 2024-09-13T07:13:32.224 INFO:teuthology.orchestra.run.smithi031.stdout:5710 write 0x7f9938c thru 0x7fa2d12 (0x9987 bytes) 2024-09-13T07:13:32.235 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.236 INFO:teuthology.orchestra.run.smithi031.stdout:5712 punch from 0x591fba0 to 0x5924fc2, (0x5422 bytes) 2024-09-13T07:13:32.236 INFO:teuthology.orchestra.run.smithi031.stdout:5713 write 0xbec8076 thru 0xbecd871 (0x57fc bytes) 2024-09-13T07:13:32.238 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:32.239+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:32.251 INFO:teuthology.orchestra.run.smithi031.stdout:5714 write 0x15852fb thru 0x158ebef (0x98f5 bytes) 2024-09-13T07:13:32.265 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.265 INFO:teuthology.orchestra.run.smithi031.stdout:5716 read 0x7a674d1 thru 0x7a70f0f (0x9a3f bytes) 2024-09-13T07:13:32.265 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.265 INFO:teuthology.orchestra.run.smithi031.stdout:5721 read 0x281129e thru 0x281bcb9 (0xaa1c bytes) 2024-09-13T07:13:32.265 INFO:teuthology.orchestra.run.smithi031.stdout:5723 clone 35 order 22 su 2097152 sc 13 2024-09-13T07:13:32.293 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:32.293 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:32.293 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:32.293 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:32.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:32.339 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:32.340+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:32.440 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:32.441+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:32.444 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:32.444 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:32.444 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:32.444 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:32.444 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:32.453 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:32.470 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:13:32.540 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:32.541+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:32.542 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone34 from 0xd32dbbd (overlap 0xd756bf) to 0x0 2024-09-13T07:13:32.565 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:32.565 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:32.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 54.528523575999998, 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:32.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:13:32.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.981614703999998, 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.572 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:13:32.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.574 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:13:32.575 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:13:32.575 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.575 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.576 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:32.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:13:32.577 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:13:32.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88519 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9804568699999994, 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017910447999999999, 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88519 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585257+0000", 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585267+0000", 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0635000000000001e-05 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585289+0000", 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1846999999999999e-05 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585299+0000", 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4949999999999997e-06 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585347+0000", 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7672e-05 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585500+0000", 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015363700000000001 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.588596+0000", 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030953669999999999 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603068+0000", 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014472063 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603106+0000", 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8435e-05 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603142+0000", 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6488e-05 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603169+0000", 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6959000000000001e-05 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.585 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:89021 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.0359799570000003, 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02102269, 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 89021 2024-09-13T07:13:32.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.587 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:32.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529734+0000", 2024-09-13T07:13:32.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529742+0000", 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8290000000000008e-06 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529757+0000", 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4868999999999999e-05 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529763+0000", 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.93e-06 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529803+0000", 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0404999999999997e-05 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529809+0000", 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531015+0000", 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012062049999999999 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531057+0000", 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1921000000000001e-05 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:32.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.534526+0000", 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034694909999999999 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550683+0000", 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016157169999999998 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550697+0000", 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3351e-05 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550731+0000", 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3997e-05 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550759+0000", 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7861e-05 2024-09-13T07:13:32.593 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.594 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.594 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.594 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95026 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.594 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8699658909999997, 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014972104999999999, 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95026 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.596 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695752+0000", 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.283e-06 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695758+0000", 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695762+0000", 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710581+0000", 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014818934000000001 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710653+0000", 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2457000000000001e-05 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710722+0000", 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8633999999999994e-05 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.599 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95027 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8699497819999999, 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015079646, 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.600 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95027 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695768+0000", 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695773+0000", 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.592e-06 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695778+0000", 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710749+0000", 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014970809 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710805+0000", 2024-09-13T07:13:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6149000000000001e-05 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710846+0000", 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0840999999999997e-05 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95028 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8699340859999998, 2024-09-13T07:13:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015231216000000001, 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95028 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695781+0000", 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695783+0000", 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9250000000000002e-06 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695789+0000", 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.835e-06 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695793+0000", 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3000000000000003e-06 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710881+0000", 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015087695999999999 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710947+0000", 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6169999999999998e-05 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.711013+0000", 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5643999999999997e-05 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95080 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:32.610 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.868332959, 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014948011000000001, 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95080 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697385+0000", 2024-09-13T07:13:32.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.187e-06 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697388+0000", 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9459999999999998e-06 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697390+0000", 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5600000000000001e-06 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712211+0000", 2024-09-13T07:13:32.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014820224 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712276+0000", 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5650999999999999e-05 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712331+0000", 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4727e-05 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.614 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95081 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8683235030000001, 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015124461000000001, 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95081 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.615 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.616 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697394+0000", 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.418e-06 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697397+0000", 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697399+0000", 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.508e-06 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712367+0000", 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01496809 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712442+0000", 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5290999999999995e-05 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712517+0000", 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4396999999999999e-05 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95082 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8683148190000001, 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015249593000000001, 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.619 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95082 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:32.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697402+0000", 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.311e-06 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697405+0000", 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697407+0000", 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712557+0000", 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015149207 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712609+0000", 2024-09-13T07:13:32.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2286e-05 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712651+0000", 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1687999999999998e-05 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95083 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3727360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8683063290000002, 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015418032, 2024-09-13T07:13:32.623 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95083 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697409+0000", 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.049e-06 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697414+0000", 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.783e-06 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697417+0000", 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.587e-06 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712683+0000", 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015266083 2024-09-13T07:13:32.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712759+0000", 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6627999999999999e-05 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712828+0000", 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8121999999999996e-05 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.627 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95084 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3776512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8682970579999996, 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015560297000000001, 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95084 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697420+0000", 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697423+0000", 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3359999999999999e-06 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697425+0000", 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.452e-06 2024-09-13T07:13:32.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712864+0000", 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015438927999999999 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712938+0000", 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3458e-05 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712979+0000", 2024-09-13T07:13:32.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1251999999999997e-05 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95085 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8682862680000003, 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015726161999999998, 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.632 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95085 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697429+0000", 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697431+0000", 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5230000000000001e-06 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697434+0000", 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:13:32.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697436+0000", 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3889999999999999e-06 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713008+0000", 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015571295000000001 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713084+0000", 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6537000000000003e-05 2024-09-13T07:13:32.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713156+0000", 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1616e-05 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95086 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3874816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8682772629999995, 2024-09-13T07:13:32.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015894697999999999, 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95086 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697438+0000", 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697440+0000", 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.159e-06 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697443+0000", 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9359999999999999e-06 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697445+0000", 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713191+0000", 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015746343999999999 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713269+0000", 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7737000000000005e-05 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713333+0000", 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4233000000000007e-05 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.640 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95087 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8682689239999997, 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01604939, 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95087 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697448+0000", 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.094e-06 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697451+0000", 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8949999999999998e-06 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697454+0000", 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.999e-06 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713361+0000", 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015907092000000001 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713433+0000", 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2606000000000002e-05 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713496+0000", 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2908000000000001e-05 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.644 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95088 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.868258784, 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016219289000000001, 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95088 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.645 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697459+0000", 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.764e-06 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697462+0000", 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4240000000000002e-06 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.647 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697464+0000", 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3989999999999998e-06 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713532+0000", 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016067403000000001 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713607+0000", 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5476999999999996e-05 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713676+0000", 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9115999999999994e-05 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95089 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8682489310000001, 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016381144, 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.649 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95089 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697468+0000", 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.734e-06 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697474+0000", 2024-09-13T07:13:32.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697479+0000", 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713703+0000", 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016224643 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:32.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4828999999999998e-05 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713848+0000", 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0029000000000003e-05 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95090 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8682331330000004, 2024-09-13T07:13:32.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016573104000000002, 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95090 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697482+0000", 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697484+0000", 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.916e-06 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697496+0000", 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697506+0000", 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0477e-05 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713903+0000", 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016396758000000001 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713981+0000", 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7892999999999994e-05 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714056+0000", 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5214000000000002e-05 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95091 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:32.657 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.868199712, 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016674768, 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95091 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697518+0000", 2024-09-13T07:13:32.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4789999999999999e-06 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697527+0000", 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3799999999999994e-06 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697532+0000", 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.660 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714080+0000", 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016547798999999998 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714135+0000", 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5294000000000003e-05 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714191+0000", 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5606999999999998e-05 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95092 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4169728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 4.8681789240000004, 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017373829, 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95092 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:32.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697536+0000", 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:32.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697540+0000", 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4879999999999999e-06 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697548+0000", 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0139999999999998e-06 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697553+0000", 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2460000000000003e-06 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.664 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714754+0000", 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017200653 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714831+0000", 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7156000000000005e-05 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714911+0000", 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9740999999999994e-05 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.666 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:32.666 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:32.666 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:32.667 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:32.668 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:32.668+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:32.767 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:32.768+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:32.815 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:32.815 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:32.824 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:32.869 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:32.870+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:32.907 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T07:13:32.936 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:32.936 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:32.936 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:32.936 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:32.936 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:32.945 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:32.945 INFO:teuthology.orchestra.run.smithi031.stdout:5724 write 0x7713fe8 thru 0x771e2fc (0xa315 bytes) 2024-09-13T07:13:32.953 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.953 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:32.953 INFO:teuthology.orchestra.run.smithi031.stdout:5728 write 0x5958f74 thru 0x595bca3 (0x2d30 bytes) 2024-09-13T07:13:32.969 INFO:teuthology.orchestra.run.smithi031.stdout:5729 trunc from 0xd32dbbd to 0x22000c1 2024-09-13T07:13:32.988 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:32.989+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:33.008 INFO:teuthology.orchestra.run.smithi031.stdout:5730 trunc from 0x22000c1 to 0xd7f885e 2024-09-13T07:13:33.014 INFO:teuthology.orchestra.run.smithi031.stdout:5731 read 0x344cf07 thru 0x3459632 (0xc72c bytes) 2024-09-13T07:13:33.014 INFO:teuthology.orchestra.run.smithi031.stdout:5732 punch from 0x2411cbd to 0x24160a1, (0x43e4 bytes) 2024-09-13T07:13:33.026 INFO:teuthology.orchestra.run.smithi031.stdout:5734 trunc from 0xd7f885e to 0x22d2e1e 2024-09-13T07:13:33.037 INFO:teuthology.orchestra.run.smithi031.stdout:5739 trunc from 0x22d2e1e to 0x3f2707c 2024-09-13T07:13:33.042 INFO:teuthology.orchestra.run.smithi031.stdout:5740 write 0x1d541b5 thru 0x1d585cd (0x4419 bytes) 2024-09-13T07:13:33.054 INFO:teuthology.orchestra.run.smithi031.stdout:5741 read 0x1b07b06 thru 0x1b0d05f (0x555a bytes) 2024-09-13T07:13:33.058 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.704994004, 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99762887700000002, 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.059 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:13:33.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351641+0000", 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99754909000000003 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351653+0000", 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2248e-05 2024-09-13T07:13:33.063 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88604 3.e 3:75ae40da:::rbd_header.117390733e87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.3361629110000006, 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20280552800000001, 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88604 2024-09-13T07:13:33.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722855+0000", 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722859+0000", 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722870+0000", 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0434999999999999e-05 2024-09-13T07:13:33.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722875+0000", 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722912+0000", 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6943000000000002e-05 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722963+0000", 2024-09-13T07:13:33.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1103000000000002e-05 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723070+0000", 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106279 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723472+0000", 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040248200000000001 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:33.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925590+0000", 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20211748900000001 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925624+0000", 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4801000000000001e-05 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925661+0000", 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6643999999999997e-05 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88605 3.e 3:75ae40da:::rbd_header.117390733e87:head [watch ping cookie 139824472318400 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 9.2859558440000001, 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15274379900000001, 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88605 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773062+0000", 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773066+0000", 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1570000000000002e-06 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773073+0000", 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5490000000000003e-06 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773080+0000", 2024-09-13T07:13:33.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773125+0000", 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4849999999999999e-05 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for rw locks", 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773190+0000", 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5567000000000004e-05 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925689+0000", 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15249922899999999 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925746+0000", 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6369999999999997e-05 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925806+0000", 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0668999999999998e-05 2024-09-13T07:13:33.074 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94703 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3366912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3738377939999999, 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027375494, 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94703 2024-09-13T07:13:33.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685180+0000", 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685183+0000", 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685189+0000", 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0730000000000001e-06 2024-09-13T07:13:33.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685193+0000", 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712474+0000", 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027280787000000001 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712507+0000", 2024-09-13T07:13:33.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2786000000000001e-05 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712556+0000", 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9203000000000003e-05 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94704 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3738204060000001, 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027564668000000001, 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.079 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94704 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:33.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685200+0000", 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685207+0000", 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:13:33.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685212+0000", 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1449999999999999e-06 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712597+0000", 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027385158 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712681+0000", 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3961000000000005e-05 2024-09-13T07:13:33.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712763+0000", 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1558999999999996e-05 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94705 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3737903810000001, 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027678611999999998, 2024-09-13T07:13:33.083 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94705 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685230+0000", 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4660000000000002e-06 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.085 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685237+0000", 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685242+0000", 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8729999999999998e-06 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712793+0000", 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027551022000000001 2024-09-13T07:13:33.086 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712835+0000", 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2537000000000003e-05 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712907+0000", 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1556000000000004e-05 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.087 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94706 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.37377284, 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027861107999999999, 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94706 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.088 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685245+0000", 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685247+0000", 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9479999999999998e-06 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685253+0000", 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685258+0000", 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.549e-06 2024-09-13T07:13:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712946+0000", 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027688252 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713033+0000", 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7472000000000005e-05 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713107+0000", 2024-09-13T07:13:33.091 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3478000000000004e-05 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94707 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3737573100000002, 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028004273, 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.092 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94707 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685263+0000", 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8449999999999999e-06 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685283+0000", 2024-09-13T07:13:33.094 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0165000000000001e-05 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685289+0000", 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7189999999999998e-06 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713138+0000", 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027849056000000001 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.095 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713189+0000", 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1801000000000001e-05 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713265+0000", 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5983999999999996e-05 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94708 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3737249719999998, 2024-09-13T07:13:33.096 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028161510000000001, 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94708 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:33.097 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.098 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685296+0000", 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-06 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685302+0000", 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685306+0000", 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.099 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713301+0000", 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027995176 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713372+0000", 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0870000000000004e-05 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713455+0000", 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2956999999999999e-05 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94709 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3737081330000001, 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357679, 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94709 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.101 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.102 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685312+0000", 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685318+0000", 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685323+0000", 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:13:33.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713505+0000", 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028182713000000002 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713592+0000", 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6397000000000006e-05 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713668+0000", 2024-09-13T07:13:33.104 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6188999999999998e-05 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94710 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3736906769999999, 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028518801, 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.105 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94710 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685327+0000", 2024-09-13T07:13:33.106 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685330+0000", 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3120000000000001e-06 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685337+0000", 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685342+0000", 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-13T07:13:33.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713700+0000", 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357838999999999 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8438999999999999e-05 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713847+0000", 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8540000000000004e-05 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.108 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94711 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3736720040000003, 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640149, 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94711 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.109 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.110 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685348+0000", 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685355+0000", 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685359+0000", 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.111 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713875+0000", 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028516414 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713931+0000", 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713987+0000", 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6084e-05 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.112 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94712 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3736551160000001, 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028779301, 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94712 2024-09-13T07:13:33.113 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.114 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685365+0000", 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685371+0000", 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7620000000000001e-06 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685375+0000", 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.42e-06 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714016+0000", 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640591 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714079+0000", 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3288999999999997e-05 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714143+0000", 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3361999999999994e-05 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94713 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3736389029999998, 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028911175000000001, 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94713 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.117 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685379+0000", 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.118 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685381+0000", 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1509999999999998e-06 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685387+0000", 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685391+0000", 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5920000000000002e-06 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.119 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714169+0000", 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028777971999999999 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714235+0000", 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5782000000000001e-05 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714291+0000", 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5525000000000001e-05 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94714 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3735986249999996, 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029006134999999999, 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94714 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.121 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685419+0000", 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685430+0000", 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015e-05 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685441+0000", 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1049e-05 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685446+0000", 2024-09-13T07:13:33.123 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714313+0000", 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866640999999998 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714368+0000", 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5024e-05 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:33.124 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7843000000000002e-05 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94715 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3735669279999998, 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029140899000000001, 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94715 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.125 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685451+0000", 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.126 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685454+0000", 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.627e-06 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685466+0000", 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2510000000000001e-05 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685472+0000", 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.127 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714453+0000", 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028980599999999999 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714524+0000", 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1186999999999995e-05 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714592+0000", 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8668000000000001e-05 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.128 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94716 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.373542016, 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029223545, 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.129 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94716 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.130 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685479+0000", 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4980000000000001e-06 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685485+0000", 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685489+0000", 2024-09-13T07:13:33.131 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714617+0000", 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029128106000000001 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714662+0000", 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.456e-05 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.132 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714700+0000", 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8281000000000002e-05 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94717 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3735251110000002, 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029398906999999998, 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.133 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94717 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:33.134 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685495+0000", 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0499999999999999e-06 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685501+0000", 2024-09-13T07:13:33.135 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9569999999999999e-06 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685506+0000", 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3249999999999998e-06 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714726+0000", 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029219218000000002 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714804+0000", 2024-09-13T07:13:33.136 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8610999999999997e-05 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714892+0000", 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8114000000000004e-05 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94718 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3735078319999996, 2024-09-13T07:13:33.137 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029526177000000001, 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94718 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.138 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685510+0000", 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685513+0000", 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4360000000000001e-06 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.139 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685519+0000", 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685523+0000", 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714918+0000", 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029395260999999999 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.140 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714985+0000", 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6737999999999997e-05 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715037+0000", 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1712999999999998e-05 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94719 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.141 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 5.3734913149999999, 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029657744, 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94719 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.142 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685529+0000", 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.143 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685535+0000", 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685539+0000", 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715062+0000", 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029522738 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.144 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715125+0000", 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3229000000000001e-05 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715185+0000", 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9441999999999998e-05 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:33.145 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:33.147 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:33.148+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:33.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:33.149 INFO:teuthology.orchestra.run.smithi031.stdout:5742 write 0x72299c3 thru 0x723830a (0xe948 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5743 punch from 0x401d935 to 0x4029926, (0xbff1 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5744 punch from 0x282d20e to 0x283a760, (0xd552 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5745 writesame 0xe434e12 thru 0xe438cb1 data_size 0x18(0x3ea0 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5746 write 0x4e85a2 thru 0x4ec346 (0x3da5 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5748 write 0xd2f095a thru 0xd2f1c5b (0x1302 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5749 write 0x4ff59c7 thru 0x4ffaaa3 (0x50dd bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5750 read 0xbb67195 thru 0xbb6b2e9 (0x4155 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5751 read 0xc4ba446 thru 0xc4bfe0e (0x59c9 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5753 punch from 0x5b9f66c to 0x5bad06e, (0xda02 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:5754 write 0xc755e42 thru 0xc7579da (0x1b99 bytes) 2024-09-13T07:13:33.150 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5756 write 0x5546ffe thru 0x5555a3d (0xea40 bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5757 writesame 0x8209854 thru 0x820ac01 data_size 0xb(0x13ae bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5758 write 0xdab1791 thru 0xdabcb41 (0xb3b1 bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5759 trunc from 0xe438cb2 to 0x9696622 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5760 read 0x1d3b24 thru 0x1e0c3a (0xd117 bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5761 punch from 0x8314368 to 0x83205fd, (0xc295 bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5762 write 0x6e2824e thru 0x6e2d50e (0x52c1 bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5763 read 0x3ee6f7f thru 0x3ef46fa (0xd77c bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5764 read 0x9511e25 thru 0x9512522 (0x6fe bytes) 2024-09-13T07:13:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:5765 trunc from 0x9696622 to 0x34bf9ec 2024-09-13T07:13:33.166 INFO:teuthology.orchestra.run.smithi031.stdout:5766 write 0x7a10b08 thru 0x7a140a7 (0x35a0 bytes) 2024-09-13T07:13:33.169 INFO:teuthology.orchestra.run.smithi031.stdout:5767 read 0x4f250bb thru 0x4f26e86 (0x1dcc bytes) 2024-09-13T07:13:33.169 INFO:teuthology.orchestra.run.smithi031.stdout:5768 writesame 0x51e8d0a thru 0x51f1ccc data_size 0x1(0x8fc3 bytes) 2024-09-13T07:13:33.189 INFO:teuthology.orchestra.run.smithi031.stdout:5769 trunc from 0x7a140a8 to 0x71bbf1f 2024-09-13T07:13:33.200 INFO:teuthology.orchestra.run.smithi031.stdout:5771 writesame 0x6549edf thru 0x65566f4 data_size 0x3(0xc816 bytes) 2024-09-13T07:13:33.211 INFO:teuthology.orchestra.run.smithi031.stdout:5772 punch from 0x50191bc to 0x5019e78, (0xcbc bytes) 2024-09-13T07:13:33.224 INFO:teuthology.orchestra.run.smithi031.stdout:5773 write 0xce22fb6 thru 0xce2d681 (0xa6cc bytes) 2024-09-13T07:13:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:5774 trunc from 0xce2d682 to 0x4aeab42 2024-09-13T07:13:33.247 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:33.248+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:33.255 INFO:teuthology.orchestra.run.smithi031.stdout:5775 write 0x5951f16 thru 0x5958a65 (0x6b50 bytes) 2024-09-13T07:13:33.259 INFO:teuthology.orchestra.run.smithi031.stdout:5776 punch from 0x1844b9a to 0x18452b0, (0x716 bytes) 2024-09-13T07:13:33.270 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:103164 3.3 e71/19 3:c0cf90fc:::rbd_header.1173308dbcae:head v 71'347, mlcod=71'345)", 2024-09-13T07:13:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:33.271240+0000", 2024-09-13T07:13:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000364247, 2024-09-13T07:13:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038515799999999997, 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271240+0000", 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271240+0000", 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271239+0000", 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271246+0000", 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7259999999999996e-06 2024-09-13T07:13:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271257+0000", 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0325e-05 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271263+0000", 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4030000000000004e-06 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271292+0000", 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9113999999999999e-05 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:33.271305+0000", 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3063e-05 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:13:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-13T07:13:33.275 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:33.275 INFO:teuthology.orchestra.run.smithi031.stdout:5777 write 0xbf47e34 thru 0xbf51e32 (0x9fff bytes) 2024-09-13T07:13:33.278 INFO:teuthology.orchestra.run.smithi031.stdout:5779 trunc from 0xbf51e33 to 0xce7568e 2024-09-13T07:13:33.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:33.284 INFO:teuthology.orchestra.run.smithi031.stdout:5781 write 0x2e4fcd1 thru 0x2e59cda (0xa00a bytes) 2024-09-13T07:13:33.286 INFO:teuthology.orchestra.run.smithi031.stdout:5784 read 0xb8d2bd8 thru 0xb8d50ea (0x2513 bytes) 2024-09-13T07:13:33.286 INFO:teuthology.orchestra.run.smithi031.stdout:5785 punch from 0x3f966ea to 0x3fa15b9, (0xaecf bytes) 2024-09-13T07:13:33.286 INFO:teuthology.orchestra.run.smithi031.stdout:5786 read 0x60032fa thru 0x600d1e6 (0x9eed bytes) 2024-09-13T07:13:33.290 INFO:teuthology.orchestra.run.smithi031.stdout:5787 trunc from 0xce7568e to 0x95f7d97 2024-09-13T07:13:33.301 INFO:teuthology.orchestra.run.smithi031.stdout:5788 read 0x29ae9e1 thru 0x29b9d88 (0xb3a8 bytes) 2024-09-13T07:13:33.301 INFO:teuthology.orchestra.run.smithi031.stdout:5789 write 0x4b6cde thru 0x4c2f31 (0xc254 bytes) 2024-09-13T07:13:33.303 INFO:teuthology.orchestra.run.smithi031.stdout:5790 read 0x1d61fca thru 0x1d63d23 (0x1d5a bytes) 2024-09-13T07:13:33.303 INFO:teuthology.orchestra.run.smithi031.stdout:5791 writesame 0x3c9c0e2 thru 0x3ca6ba7 data_size 0x1(0xaac6 bytes) 2024-09-13T07:13:33.330 INFO:teuthology.orchestra.run.smithi031.stdout:5792 trunc from 0x95f7d97 to 0x5f316b0 2024-09-13T07:13:33.340 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.340 INFO:teuthology.orchestra.run.smithi031.stdout:5795 read 0x3ff76dc thru 0x4004a64 (0xd389 bytes) 2024-09-13T07:13:33.345 INFO:teuthology.orchestra.run.smithi031.stdout:5796 write 0x8720890 thru 0x8724ef8 (0x4669 bytes) 2024-09-13T07:13:33.347 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:33.348+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:33.360 INFO:teuthology.orchestra.run.smithi031.stdout:5798 write 0xe48c1d7 thru 0xe4963f1 (0xa21b bytes) 2024-09-13T07:13:33.364 INFO:teuthology.orchestra.run.smithi031.stdout:5799 punch from 0x5dd5204 to 0x5ddff5e, (0xad5a bytes) 2024-09-13T07:13:33.364 INFO:teuthology.orchestra.run.smithi031.stdout:5800 read 0x5f4fd27 thru 0x5f5cc04 (0xcede bytes) 2024-09-13T07:13:33.364 INFO:teuthology.orchestra.run.smithi031.stdout:5801 read 0x1878334 thru 0x18787d0 (0x49d bytes) 2024-09-13T07:13:33.364 INFO:teuthology.orchestra.run.smithi031.stdout:5802 writesame 0x3e6ab6c thru 0x3e6f095 data_size 0x27(0x452a bytes) 2024-09-13T07:13:33.365 INFO:teuthology.orchestra.run.smithi031.stdout:5803 writesame 0xae27641 thru 0xae2fdbc data_size 0x5c(0x877c bytes) 2024-09-13T07:13:33.368 INFO:teuthology.orchestra.run.smithi031.stdout:5804 write 0xb932eda thru 0xb940c2c (0xdd53 bytes) 2024-09-13T07:13:33.372 INFO:teuthology.orchestra.run.smithi031.stdout:5806 read 0xb38af71 thru 0xb397037 (0xc0c7 bytes) 2024-09-13T07:13:33.372 INFO:teuthology.orchestra.run.smithi031.stdout:5807 writesame 0x4ae6136 thru 0x4ae92d1 data_size 0x32(0x319c bytes) 2024-09-13T07:13:33.373 INFO:teuthology.orchestra.run.smithi031.stdout:5809 read 0x976efa8 thru 0x9772c60 (0x3cb9 bytes) 2024-09-13T07:13:33.373 INFO:teuthology.orchestra.run.smithi031.stdout:5810 write 0x32b1017 thru 0x32c0669 (0xf653 bytes) 2024-09-13T07:13:33.375 INFO:teuthology.orchestra.run.smithi031.stdout:5811 write 0x5778ddc thru 0x578348c (0xa6b1 bytes) 2024-09-13T07:13:33.377 INFO:teuthology.orchestra.run.smithi031.stdout:5813 write 0x288e342 thru 0x289765a (0x9319 bytes) 2024-09-13T07:13:33.378 INFO:teuthology.orchestra.run.smithi031.stdout:5814 read 0x9c21161 thru 0x9c26acf (0x596f bytes) 2024-09-13T07:13:33.378 INFO:teuthology.orchestra.run.smithi031.stdout:5817 punch from 0xbcf9a69 to 0xbd01706, (0x7c9d bytes) 2024-09-13T07:13:33.378 INFO:teuthology.orchestra.run.smithi031.stdout:5818 write 0xc4552f3 thru 0xc4588a9 (0x35b7 bytes) 2024-09-13T07:13:33.389 INFO:teuthology.orchestra.run.smithi031.stdout:5820 trunc from 0xe4963f2 to 0x6ae5350 2024-09-13T07:13:33.403 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.403 INFO:teuthology.orchestra.run.smithi031.stdout:5823 read 0x1062fdc thru 0x10701ea (0xd20f bytes) 2024-09-13T07:13:33.404 INFO:teuthology.orchestra.run.smithi031.stdout:5824 punch from 0x59a3c78 to 0x59a4b19, (0xea1 bytes) 2024-09-13T07:13:33.404 INFO:teuthology.orchestra.run.smithi031.stdout:5826 read 0x364cf69 thru 0x3653dcf (0x6e67 bytes) 2024-09-13T07:13:33.404 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:33.404 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:33.411 INFO:teuthology.orchestra.run.smithi031.stdout:5827 write 0x8386d8a thru 0x838a30f (0x3586 bytes) 2024-09-13T07:13:33.414 INFO:teuthology.orchestra.run.smithi031.stdout:5828 read 0x3792c30 thru 0x379cd34 (0xa105 bytes) 2024-09-13T07:13:33.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:33.416 INFO:teuthology.orchestra.run.smithi031.stdout:5829 read 0x724d20f thru 0x725bf1d (0xed0f bytes) 2024-09-13T07:13:33.416 INFO:teuthology.orchestra.run.smithi031.stdout:5830 write 0x4ec7598 thru 0x4ed5e53 (0xe8bc bytes) 2024-09-13T07:13:33.427 INFO:teuthology.orchestra.run.smithi031.stdout:5831 write 0xcf9ecb3 thru 0xcfa4a78 (0x5dc6 bytes) 2024-09-13T07:13:33.439 INFO:teuthology.orchestra.run.smithi031.stdout:5834 writesame 0xebc3fd6 thru 0xebcf218 data_size 0x1(0xb243 bytes) 2024-09-13T07:13:33.439 INFO:teuthology.orchestra.run.smithi031.stdout:5836 read 0xcb5c21c thru 0xcb66757 (0xa53c bytes) 2024-09-13T07:13:33.439 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.439 INFO:teuthology.orchestra.run.smithi031.stdout:5838 read 0x36da814 thru 0x36e26d1 (0x7ebe bytes) 2024-09-13T07:13:33.443 INFO:teuthology.orchestra.run.smithi031.stdout:5839 trunc from 0xebcf219 to 0xb4d957e 2024-09-13T07:13:33.448 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:33.449+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:33.450 INFO:teuthology.orchestra.run.smithi031.stdout:5841 write 0x46e5279 thru 0x46e9c26 (0x49ae bytes) 2024-09-13T07:13:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:5842 read 0x35744a7 thru 0x35825c4 (0xe11e bytes) 2024-09-13T07:13:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:5843 read 0xa6395f5 thru 0xa63ab92 (0x159e bytes) 2024-09-13T07:13:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:5844 read 0x6991557 thru 0x699e799 (0xd243 bytes) 2024-09-13T07:13:33.462 INFO:teuthology.orchestra.run.smithi031.stdout:5845 write 0xd9135d6 thru 0xd918b45 (0x5570 bytes) 2024-09-13T07:13:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:5846 trunc from 0xd918b46 to 0x4a00adb 2024-09-13T07:13:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:5848 read 0x49a4cc8 thru 0x49a9ac9 (0x4e02 bytes) 2024-09-13T07:13:33.498 INFO:teuthology.orchestra.run.smithi031.stdout:5849 write 0x4defec5 thru 0x4df9469 (0x95a5 bytes) 2024-09-13T07:13:33.517 INFO:teuthology.orchestra.run.smithi031.stdout:5850 write 0xd390c0b thru 0xd399ac7 (0x8ebd bytes) 2024-09-13T07:13:33.520 INFO:teuthology.orchestra.run.smithi031.stdout:5852 read 0x85b5f2d thru 0x85bead8 (0x8bac bytes) 2024-09-13T07:13:33.520 INFO:teuthology.orchestra.run.smithi031.stdout:5854 read 0x5e6f226 thru 0x5e74891 (0x566c bytes) 2024-09-13T07:13:33.520 INFO:teuthology.orchestra.run.smithi031.stdout:5856 writesame 0x5e1a73c thru 0x5e29660 data_size 0x1(0xef25 bytes) 2024-09-13T07:13:33.549 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:33.550+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:33.550 INFO:teuthology.orchestra.run.smithi031.stdout:5857 writesame 0x77fd17 thru 0x788184 data_size 0x43(0x846e bytes) 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51075 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021587404000002, 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081563839999999999, 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51075 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535172+0000", 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-09-13T07:13:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535178+0000", 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535183+0000", 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000001e-06 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616726+0000", 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081543051000000005 2024-09-13T07:13:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616734+0000", 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51076 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021570403999998, 2024-09-13T07:13:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081579246999999994, 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51076 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535189+0000", 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9709999999999998e-06 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535195+0000", 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0739999999999998e-06 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535199+0000", 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5549999999999996e-06 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616759+0000", 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081559926000000005 2024-09-13T07:13:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616766+0000", 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51077 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021553785999998, 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081595972000000003, 2024-09-13T07:13:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51077 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535203+0000", 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535210+0000", 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535217+0000", 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3490000000000003e-06 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535222+0000", 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616791+0000", 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081568733000000004 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616799+0000", 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.405e-06 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51078 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021530108, 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081603360999999999, 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51078 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535229+0000", 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535236+0000", 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.489e-06 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535240+0000", 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616823+0000", 2024-09-13T07:13:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081582882999999995 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616831+0000", 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2280000000000001e-06 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51079 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021512654999999, 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081617985000000004, 2024-09-13T07:13:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51079 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535244+0000", 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535247+0000", 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535253+0000", 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2249999999999997e-06 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535258+0000", 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8899999999999998e-06 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616855+0000", 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081597307999999993 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616863+0000", 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6580000000000002e-06 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51080 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021494724, 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081642305999999998, 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51080 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535262+0000", 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535264+0000", 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535275+0000", 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0936e-05 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535281+0000", 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616889+0000", 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081608705000000004 2024-09-13T07:13:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616905+0000", 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5398999999999999e-05 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51081 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021471736999999, 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081649656000000001, 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51081 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535285+0000", 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535288+0000", 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535294+0000", 2024-09-13T07:13:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535299+0000", 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616928+0000", 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081629060000000003 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616935+0000", 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.968e-06 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51082 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021453726000001, 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081660766999999995, 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51082 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535303+0000", 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535305+0000", 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535312+0000", 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-13T07:13:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535317+0000", 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616957+0000", 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081640502000000004 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616964+0000", 2024-09-13T07:13:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51083 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021436091999998, 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081673832000000002, 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51083 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535323+0000", 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535329+0000", 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2650000000000002e-06 2024-09-13T07:13:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535334+0000", 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616988+0000", 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081653901000000001 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616995+0000", 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1980000000000004e-06 2024-09-13T07:13:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51084 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021418769, 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081686317999999994, 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51084 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535338+0000", 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535340+0000", 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0640000000000001e-06 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535347+0000", 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535351+0000", 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617017+0000", 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081665777999999994 2024-09-13T07:13:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617025+0000", 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5789999999999998e-06 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51085 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021401387000001, 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081702650000000002, 2024-09-13T07:13:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51085 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535358+0000", 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535364+0000", 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535369+0000", 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617051+0000", 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081681576000000006 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617059+0000", 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51086 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.021383592999999, 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081714234999999996, 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51086 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535373+0000", 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535376+0000", 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535382+0000", 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3899999999999998e-06 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535387+0000", 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7559999999999999e-06 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617081+0000", 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081694514999999995 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617088+0000", 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9362912889999997, 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82619132900000003, 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620465+0000", 2024-09-13T07:13:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620483+0000", 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620497+0000", 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4248e-05 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620505+0000", 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9559999999999997e-06 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620563+0000", 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7667999999999997e-05 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620573+0000", 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9289999999999993e-06 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446642+0000", 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82606955599999998 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446657+0000", 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4932e-05 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9362257109999996, 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82648916299999997, 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620531+0000", 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620535+0000", 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3959999999999999e-06 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620545+0000", 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5030000000000003e-06 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620551+0000", 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4690000000000002e-06 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620596+0000", 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4821000000000003e-05 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620604+0000", 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6359999999999994e-06 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447008+0000", 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82640484000000003 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447021+0000", 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2320999999999999e-05 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.842509315, 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73282608599999999, 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714247+0000", 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714303+0000", 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5475e-05 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714338+0000", 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5611000000000002e-05 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714345+0000", 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-13T07:13:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714410+0000", 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5223999999999998e-05 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714417+0000", 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447059+0000", 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73264227900000001 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447074+0000", 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5078e-05 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8423315450000004, 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73271206499999997, 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714425+0000", 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714430+0000", 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714441+0000", 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0645999999999999e-05 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714450+0000", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714489+0000", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8974000000000003e-05 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714494+0000", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447125+0000", 2024-09-13T07:13:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73263175700000005 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447138+0000", 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2532e-05 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8371876880000002, 2024-09-13T07:13:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72759390000000002, 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719569+0000", 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719586+0000", 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7397999999999998e-05 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719597+0000", 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0368e-05 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719603+0000", 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719649+0000", 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5818000000000001e-05 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719658+0000", 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0680000000000003e-06 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447157+0000", 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72749929599999996 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447164+0000", 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3659999999999997e-06 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8369854910000001, 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72744244800000002, 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719771+0000", 2024-09-13T07:13:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719775+0000", 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719782+0000", 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3949999999999998e-06 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719789+0000", 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4719999999999998e-06 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719830+0000", 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1140000000000003e-05 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719841+0000", 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0808999999999999e-05 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447202+0000", 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72736066499999996 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447214+0000", 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2472e-05 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8236798079999996, 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71610967800000003, 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733095+0000", 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733107+0000", 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1902999999999999e-05 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733115+0000", 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733154+0000", 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8501000000000003e-05 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733159+0000", 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2449999999999998e-06 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447255+0000", 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71409564299999995 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447366+0000", 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111273 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447510+0000", 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014420399999999999 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448696+0000", 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001186246 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449068+0000", 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037139399999999999 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449145+0000", 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449187+0000", 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2407999999999998e-05 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [write 0~0] snapc 22=[22] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8235240260000003, 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71614144199999996, 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733232+0000", 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733237+0000", 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2880000000000003e-06 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733249+0000", 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2259000000000001e-05 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733256+0000", 2024-09-13T07:13:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.742e-06 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733284+0000", 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8708e-05 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733294+0000", 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4089999999999992e-06 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447905+0000", 2024-09-13T07:13:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71461088299999997 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448019+0000", 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011461800000000001 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448195+0000", 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017585699999999999 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448989+0000", 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079339899999999997 2024-09-13T07:13:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449320+0000", 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000331825 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449350+0000", 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0111999999999999e-05 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449375+0000", 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4278e-05 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:33.645 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:5858 write 0x46ad483 thru 0x46b2563 (0x50e1 bytes) 2024-09-13T07:13:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:5859 punch from 0xf02e59 to 0xf1289a, (0xfa41 bytes) 2024-09-13T07:13:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:5862 read 0xa93eae9 thru 0xa94e6ae (0xfbc6 bytes) 2024-09-13T07:13:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:5864 punch from 0xb93d679 to 0xb944e50, (0x77d7 bytes) 2024-09-13T07:13:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:5865 writesame 0x9b7f17a thru 0x9b87b8d data_size 0x2(0x8a14 bytes) 2024-09-13T07:13:33.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5867 writesame 0xad304aa thru 0xad3bde3 data_size 0x15(0xb93a bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5868 read 0xbbdc0ac thru 0xbbea98e (0xe8e3 bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5870 writesame 0xb3dccda thru 0xb3de290 data_size 0x11(0x15b7 bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5871 read 0x6b37cfd thru 0x6b3a357 (0x265b bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5873 write 0x3134493 thru 0x3139205 (0x4d73 bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5874 read 0xb0acae0 thru 0xb0b6235 (0x9756 bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5875 write 0xbfaad3a thru 0xbfb29aa (0x7c71 bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5876 read 0x9e94a67 thru 0x9e96c1d (0x21b7 bytes) 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5877 trunc from 0xd399ac8 to 0x71bd844 2024-09-13T07:13:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:5878 trunc from 0x71bd844 to 0x319436d 2024-09-13T07:13:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:5879 write 0x8ffe5b6 thru 0x9005c18 (0x7663 bytes) 2024-09-13T07:13:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:5881 writesame 0x769fdfa thru 0x76adbe9 data_size 0x4(0xddf0 bytes) 2024-09-13T07:13:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:5885 read 0x3cdabf2 thru 0x3cdd846 (0x2c55 bytes) 2024-09-13T07:13:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:5888 read 0xff1f04 thru 0xff8117 (0x6214 bytes) 2024-09-13T07:13:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:5889 read 0x11b85d6 thru 0x11b8ea9 (0x8d4 bytes) 2024-09-13T07:13:33.651 INFO:teuthology.orchestra.run.smithi031.stdout:5890 write 0x4439d71 thru 0x44453e0 (0xb670 bytes) 2024-09-13T07:13:33.652 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:33.652+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:33.653 INFO:teuthology.orchestra.run.smithi031.stdout:5891 read 0x48f391d thru 0x4901a4d (0xe131 bytes) 2024-09-13T07:13:33.653 INFO:teuthology.orchestra.run.smithi031.stdout:5893 punch from 0xa33f64 to 0xa43245, (0xf2e1 bytes) 2024-09-13T07:13:33.662 INFO:teuthology.orchestra.run.smithi031.stdout:5894 writesame 0xc53116c thru 0xc5404bb data_size 0x88(0xf350 bytes) 2024-09-13T07:13:33.670 INFO:teuthology.orchestra.run.smithi031.stdout:5895 write 0xeb4416b thru 0xeb5211c (0xdfb2 bytes) 2024-09-13T07:13:33.673 INFO:teuthology.orchestra.run.smithi031.stdout:5896 read 0x5d07ab4 thru 0x5d0cd19 (0x5266 bytes) 2024-09-13T07:13:33.673 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.673 INFO:teuthology.orchestra.run.smithi031.stdout:5899 punch from 0x205d298 to 0x2061d7e, (0x4ae6 bytes) 2024-09-13T07:13:33.673 INFO:teuthology.orchestra.run.smithi031.stdout:5900 punch from 0xafcb73c to 0xafcd8e4, (0x21a8 bytes) 2024-09-13T07:13:33.673 INFO:teuthology.orchestra.run.smithi031.stdout:5902 write 0x11603fc thru 0x1163e20 (0x3a25 bytes) 2024-09-13T07:13:33.675 INFO:teuthology.orchestra.run.smithi031.stdout:5903 read 0x63abad9 thru 0x63ba8ba (0xede2 bytes) 2024-09-13T07:13:33.675 INFO:teuthology.orchestra.run.smithi031.stdout:5904 read 0x7727f84 thru 0x772da7e (0x5afb bytes) 2024-09-13T07:13:33.675 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.675 INFO:teuthology.orchestra.run.smithi031.stdout:5907 write 0x4139c8 thru 0x42282f (0xee68 bytes) 2024-09-13T07:13:33.677 INFO:teuthology.orchestra.run.smithi031.stdout:5908 writesame 0x6b0edff thru 0x6b111e7 data_size 0x1(0x23e9 bytes) 2024-09-13T07:13:33.699 INFO:teuthology.orchestra.run.smithi031.stdout:5911 trunc from 0xeb5211d to 0x16e2741 2024-09-13T07:13:33.721 INFO:teuthology.orchestra.run.smithi031.stdout:5912 read 0x34bed2 thru 0x3540eb (0x821a bytes) 2024-09-13T07:13:33.730 INFO:teuthology.orchestra.run.smithi031.stdout:5915 trunc from 0x16e2741 to 0x8123b65 2024-09-13T07:13:33.735 INFO:teuthology.orchestra.run.smithi031.stdout:5916 punch from 0x80470 to 0x81458, (0xfe8 bytes) 2024-09-13T07:13:33.735 INFO:teuthology.orchestra.run.smithi031.stdout:5919 punch from 0x1779413 to 0x177df68, (0x4b55 bytes) 2024-09-13T07:13:33.735 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.735 INFO:teuthology.orchestra.run.smithi031.stdout:5923 punch from 0xb3539d to 0xb41203, (0xbe66 bytes) 2024-09-13T07:13:33.735 INFO:teuthology.orchestra.run.smithi031.stdout:5925 read 0x6bd5aa1 thru 0x6bde2d1 (0x8831 bytes) 2024-09-13T07:13:33.736 INFO:teuthology.orchestra.run.smithi031.stdout:5926 read 0x6e625eb thru 0x6e6dacd (0xb4e3 bytes) 2024-09-13T07:13:33.741 INFO:teuthology.orchestra.run.smithi031.stdout:5929 write 0x8c11dce thru 0x8c1658e (0x47c1 bytes) 2024-09-13T07:13:33.744 INFO:teuthology.orchestra.run.smithi031.stdout:5930 write 0x6554efa thru 0x655ee6d (0x9f74 bytes) 2024-09-13T07:13:33.747 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.747 INFO:teuthology.orchestra.run.smithi031.stdout:5933 writesame 0x1eccd37 thru 0x1edb840 data_size 0x37(0xeb0a bytes) 2024-09-13T07:13:33.749 INFO:teuthology.orchestra.run.smithi031.stdout:5934 read 0x8da46e thru 0x8e3b4b (0x96de bytes) 2024-09-13T07:13:33.749 INFO:teuthology.orchestra.run.smithi031.stdout:5935 read 0x359a246 thru 0x35a1ca5 (0x7a60 bytes) 2024-09-13T07:13:33.752 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:33.753+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:33.755 INFO:teuthology.orchestra.run.smithi031.stdout:5936 write 0x93c23bb thru 0x93cd26a (0xaeb0 bytes) 2024-09-13T07:13:33.758 INFO:teuthology.orchestra.run.smithi031.stdout:5939 write 0x1a1c656 thru 0x1a20a5b (0x4406 bytes) 2024-09-13T07:13:33.759 INFO:teuthology.orchestra.run.smithi031.stdout:5940 read 0x2dcca00 thru 0x2dce105 (0x1706 bytes) 2024-09-13T07:13:33.759 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.760 INFO:teuthology.orchestra.run.smithi031.stdout:5942 punch from 0xca824d to 0xcb3f5f, (0xbd12 bytes) 2024-09-13T07:13:33.771 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:33.771 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:33.771 INFO:teuthology.orchestra.run.smithi031.stdout:5943 write 0xe472238 thru 0xe4746d9 (0x24a2 bytes) 2024-09-13T07:13:33.775 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.775 INFO:teuthology.orchestra.run.smithi031.stdout:5945 punch from 0x923fa97 to 0x9243097, (0x3600 bytes) 2024-09-13T07:13:33.775 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.775 INFO:teuthology.orchestra.run.smithi031.stdout:5948 writesame 0x16b9969 thru 0x16c7738 data_size 0x9c(0xddd0 bytes) 2024-09-13T07:13:33.777 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.777 INFO:teuthology.orchestra.run.smithi031.stdout:5950 writesame 0xbe31a36 thru 0xbe359b8 data_size 0x1(0x3f83 bytes) 2024-09-13T07:13:33.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:33.787 INFO:teuthology.orchestra.run.smithi031.stdout:5951 writesame 0x7085bc5 thru 0x70886ea data_size 0x15(0x2b26 bytes) 2024-09-13T07:13:33.791 INFO:teuthology.orchestra.run.smithi031.stdout:5952 read 0xde99cf0 thru 0xdea034d (0x665e bytes) 2024-09-13T07:13:33.791 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.791 INFO:teuthology.orchestra.run.smithi031.stdout:5954 read 0x311f8b0 thru 0x3124189 (0x48da bytes) 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:5957 punch from 0x7314d3f to 0x7323b21, (0xede2 bytes) 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:5959 read 0xa524db8 thru 0xa530b46 (0xbd8f bytes) 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:5962 read 0x359412c thru 0x359b7f9 (0x76ce bytes) 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:5964 read 0x3b10ccd thru 0x3b140fb (0x342f bytes) 2024-09-13T07:13:33.792 INFO:teuthology.orchestra.run.smithi031.stdout:5965 write 0x600efed thru 0x601bf70 (0xcf84 bytes) 2024-09-13T07:13:33.795 INFO:teuthology.orchestra.run.smithi031.stdout:5968 write 0x95828ac thru 0x9590160 (0xd8b5 bytes) 2024-09-13T07:13:33.810 INFO:teuthology.orchestra.run.smithi031.stdout:5969 trunc from 0xe4746da to 0x3952029 2024-09-13T07:13:33.829 INFO:teuthology.orchestra.run.smithi031.stdout:5971 read 0x1c76b3b thru 0x1c79abb (0x2f81 bytes) 2024-09-13T07:13:33.846 INFO:teuthology.orchestra.run.smithi031.stdout:5972 write 0xce0b771 thru 0xce15fef (0xa87f bytes) 2024-09-13T07:13:33.849 INFO:teuthology.orchestra.run.smithi031.stdout:5973 punch from 0x36a2dd6 to 0x36aaa0c, (0x7c36 bytes) 2024-09-13T07:13:33.849 INFO:teuthology.orchestra.run.smithi031.stdout:5974 write 0x8d5d73a thru 0x8d6bcce (0xe595 bytes) 2024-09-13T07:13:33.853 INFO:teuthology.orchestra.run.smithi031.stdout:5976 read 0xc924f5f thru 0xc9254e7 (0x589 bytes) 2024-09-13T07:13:33.853 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:33.854+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:33.856 INFO:teuthology.orchestra.run.smithi031.stdout:5979 trunc from 0xce15ff0 to 0x9a1cea5 2024-09-13T07:13:33.867 INFO:teuthology.orchestra.run.smithi031.stdout:5980 read 0x50dd6d6 thru 0x50e7992 (0xa2bd bytes) 2024-09-13T07:13:33.872 INFO:teuthology.orchestra.run.smithi031.stdout:5981 trunc from 0x9a1cea5 to 0xdf2e6a1 2024-09-13T07:13:33.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:33.894 INFO:teuthology.orchestra.run.smithi031.stdout:5982 trunc from 0xdf2e6a1 to 0x1288a66 2024-09-13T07:13:33.913 INFO:teuthology.orchestra.run.smithi031.stdout:5983 read 0x22d4ee thru 0x236071 (0x8b84 bytes) 2024-09-13T07:13:33.914 INFO:teuthology.orchestra.run.smithi031.stdout:5984 punch from 0xed7398 to 0xee492a, (0xd592 bytes) 2024-09-13T07:13:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:5986 trunc from 0x1288a66 to 0x5faa94a 2024-09-13T07:13:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:33.927 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.927 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.927 INFO:teuthology.orchestra.run.smithi031.stdout:5989 read 0x20839da thru 0x209077f (0xcda6 bytes) 2024-09-13T07:13:33.928 INFO:teuthology.orchestra.run.smithi031.stdout:5990 punch from 0x49c13b2 to 0x49cf5f9, (0xe247 bytes) 2024-09-13T07:13:33.933 INFO:teuthology.orchestra.run.smithi031.stdout:5991 write 0x60cb3c5 thru 0x60d63e7 (0xb023 bytes) 2024-09-13T07:13:33.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:33.944 INFO:teuthology.orchestra.run.smithi031.stdout:5992 writesame 0x8343348 thru 0x834d7bd data_size 0x9(0xa476 bytes) 2024-09-13T07:13:33.944 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T07:13:33.944 INFO:teuthology.orchestra.run.smithi031.stdout:5994 read 0x33d58a7 thru 0x33de58d (0x8ce7 bytes) 2024-09-13T07:13:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:5995 write 0xc90f928 thru 0xc91ca34 (0xd10d bytes) 2024-09-13T07:13:33.956 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:33.957+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:33.968 INFO:teuthology.orchestra.run.smithi031.stdout:5997 trunc from 0xc91ca35 to 0x4e6ea9d 2024-09-13T07:13:33.982 INFO:teuthology.orchestra.run.smithi031.stdout:5998 trunc from 0x4e6ea9d to 0x33844d6 2024-09-13T07:13:33.992 INFO:teuthology.orchestra.run.smithi031.stdout:6000 punch from 0x289e47e to 0x28a15e8, (0x316a bytes) 2024-09-13T07:13:34.006 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T07:13:34.033 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:34.033 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:34.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:34.054 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:34.055+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:34.068 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:34.068 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:34.071 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:34.071 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:34.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:34.089 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:34.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:34.128 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:34.128 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:34.146 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:34.146 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:34.157 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:34.157+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:34.162 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:34.201 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:34.201 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:34.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:34.258 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:34.259+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:34.356 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:34.356 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:34.358 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:34.359+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:34.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:34.460 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:34.461+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:34.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:13:34.524 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:34.525 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:34.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:34.575 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:34.576+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:34.661 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:34.662+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:34.686 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:34.686 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:34.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:34.762 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:34.763+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:34.801 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:34.801 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:34.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:34.863 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:34.864+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:34.916 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:34.916 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:34.925 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:34.966 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:34.967+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:35.028 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:35.028 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:35.037 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:35.080 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:35.081+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:35.138 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:35.139 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:35.147 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:35.166 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:35.167+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:35.267 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:35.268+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:35.289 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:35.289 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:35.299 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:35.383 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:35.384+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:35.442 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:35.442 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:35.450 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:35.468 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:35.469+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:35.485 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:13:35.569 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:35.570+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:35.599 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:35.600 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:35.608 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:35.692 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:35.693+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:35.751 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:35.751 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:35.760 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:35.804 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:35.802+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:35.873 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:35.874+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:35.906 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:35.906 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:35.914 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:35.974 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:35.975+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:36.056 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:36.056 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:36.064 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:36.074 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:36.075+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:36.167 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:36.167 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:36.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:36.176 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:36.176 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:36.176+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:36.276 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:36.277+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:36.312 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:36.320 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:36.329 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:36.329 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:36.345 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:36.345 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:36.361 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:36.377 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:36.378+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:36.377 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:36.393 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:36.410 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:36.426 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:36.426 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:36.471 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:36.472 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:36.472 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:36.472 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:36.472 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:36.478 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:36.479+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:36.480 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:36.483 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:36.483 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:36.484 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-13T07:13:36.484 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:13:36.484 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:13:36.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-13T07:13:36.579 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:36.579+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.237168991000001, 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99762887700000002, 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.591 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:13:36.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:13:36.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351641+0000", 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99754909000000003 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351653+0000", 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2248e-05 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88604 3.e 3:75ae40da:::rbd_header.117390733e87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.868337898, 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20280552800000001, 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88604 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.596 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722855+0000", 2024-09-13T07:13:36.597 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722859+0000", 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722870+0000", 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0434999999999999e-05 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.598 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722875+0000", 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722912+0000", 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6943000000000002e-05 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.599 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722963+0000", 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1103000000000002e-05 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723070+0000", 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106279 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.600 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723472+0000", 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040248200000000001 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925590+0000", 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20211748900000001 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.601 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925624+0000", 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4801000000000001e-05 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925661+0000", 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6643999999999997e-05 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88605 3.e 3:75ae40da:::rbd_header.117390733e87:head [watch ping cookie 139824472318400 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:36.602 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 12.818130831, 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15274379900000001, 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88605 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.603 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773062+0000", 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773066+0000", 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1570000000000002e-06 2024-09-13T07:13:36.604 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773073+0000", 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5490000000000003e-06 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773080+0000", 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773125+0000", 2024-09-13T07:13:36.605 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4849999999999999e-05 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for rw locks", 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773190+0000", 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5567000000000004e-05 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925689+0000", 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15249922899999999 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925746+0000", 2024-09-13T07:13:36.606 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6369999999999997e-05 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925806+0000", 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0668999999999998e-05 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94703 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3366912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9060127809999994, 2024-09-13T07:13:36.607 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027375494, 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94703 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.608 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685180+0000", 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685183+0000", 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.609 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685189+0000", 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0730000000000001e-06 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685193+0000", 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712474+0000", 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027280787000000001 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.610 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712507+0000", 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2786000000000001e-05 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712556+0000", 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9203000000000003e-05 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94704 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9059953929999995, 2024-09-13T07:13:36.611 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027564668000000001, 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94704 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.612 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685200+0000", 2024-09-13T07:13:36.613 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685207+0000", 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685212+0000", 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1449999999999999e-06 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.614 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712597+0000", 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027385158 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712681+0000", 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3961000000000005e-05 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712763+0000", 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1558999999999996e-05 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.615 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94705 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9059653680000004, 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027678611999999998, 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94705 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.616 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.617 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685230+0000", 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4660000000000002e-06 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685237+0000", 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685242+0000", 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8729999999999998e-06 2024-09-13T07:13:36.618 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712793+0000", 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027551022000000001 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712835+0000", 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2537000000000003e-05 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.619 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712907+0000", 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1556000000000004e-05 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94706 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9059478270000003, 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027861107999999999, 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.620 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94706 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.621 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685245+0000", 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685247+0000", 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9479999999999998e-06 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685253+0000", 2024-09-13T07:13:36.622 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685258+0000", 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.549e-06 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712946+0000", 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027688252 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.623 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713033+0000", 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7472000000000005e-05 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713107+0000", 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3478000000000004e-05 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94707 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:36.624 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9059322969999997, 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028004273, 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94707 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.625 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685263+0000", 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8449999999999999e-06 2024-09-13T07:13:36.626 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685283+0000", 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0165000000000001e-05 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685289+0000", 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7189999999999998e-06 2024-09-13T07:13:36.627 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.628 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.628 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713138+0000", 2024-09-13T07:13:36.628 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027849056000000001 2024-09-13T07:13:36.628 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.628 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713189+0000", 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1801000000000001e-05 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713265+0000", 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5983999999999996e-05 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.629 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94708 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9058999589999992, 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028161510000000001, 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94708 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:36.630 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.631 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685296+0000", 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-06 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685302+0000", 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685306+0000", 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.632 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713301+0000", 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027995176 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713372+0000", 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0870000000000004e-05 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713455+0000", 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2956999999999999e-05 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.633 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94709 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9058831200000004, 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357679, 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94709 2024-09-13T07:13:36.634 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:36.635 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685312+0000", 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685318+0000", 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.636 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685323+0000", 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713505+0000", 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028182713000000002 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713592+0000", 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6397000000000006e-05 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.637 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713668+0000", 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6188999999999998e-05 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94710 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9058656640000002, 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028518801, 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.638 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94710 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.639 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685327+0000", 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685330+0000", 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3120000000000001e-06 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.640 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685337+0000", 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685342+0000", 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713700+0000", 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357838999999999 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.641 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8438999999999999e-05 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713847+0000", 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8540000000000004e-05 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94711 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9058469910000007, 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640149, 2024-09-13T07:13:36.642 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94711 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.643 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685348+0000", 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.644 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685355+0000", 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685359+0000", 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713875+0000", 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028516414 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.645 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713931+0000", 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713987+0000", 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6084e-05 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94712 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:36.646 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9058301029999996, 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028779301, 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94712 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.647 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.648 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685365+0000", 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685371+0000", 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7620000000000001e-06 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.649 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685375+0000", 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.42e-06 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714016+0000", 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640591 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714079+0000", 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3288999999999997e-05 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.650 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714143+0000", 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3361999999999994e-05 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94713 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9058138899999992, 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028911175000000001, 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.651 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94713 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685379+0000", 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685381+0000", 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1509999999999998e-06 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.653 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685387+0000", 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685391+0000", 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5920000000000002e-06 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714169+0000", 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028777971999999999 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.654 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714235+0000", 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5782000000000001e-05 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714291+0000", 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5525000000000001e-05 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94714 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.655 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9057736120000008, 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029006134999999999, 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94714 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.656 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685419+0000", 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685430+0000", 2024-09-13T07:13:36.657 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015e-05 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685441+0000", 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1049e-05 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685446+0000", 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.658 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714313+0000", 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866640999999998 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714368+0000", 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5024e-05 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7843000000000002e-05 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.659 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94715 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9057419150000001, 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029140899000000001, 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94715 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.660 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685451+0000", 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.661 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685454+0000", 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.627e-06 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685466+0000", 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2510000000000001e-05 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685472+0000", 2024-09-13T07:13:36.662 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714453+0000", 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028980599999999999 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714524+0000", 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1186999999999995e-05 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.663 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714592+0000", 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8668000000000001e-05 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94716 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9057170029999995, 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029223545, 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.664 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94716 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.665 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685479+0000", 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4980000000000001e-06 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.666 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685485+0000", 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685489+0000", 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714617+0000", 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029128106000000001 2024-09-13T07:13:36.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714662+0000", 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.456e-05 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714700+0000", 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8281000000000002e-05 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.668 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94717 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9057000980000005, 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029398906999999998, 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94717 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.669 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.670 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685495+0000", 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0499999999999999e-06 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685501+0000", 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9569999999999999e-06 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685506+0000", 2024-09-13T07:13:36.671 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3249999999999998e-06 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714726+0000", 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029219218000000002 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714804+0000", 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8610999999999997e-05 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.672 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714892+0000", 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8114000000000004e-05 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94718 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9056828190000008, 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029526177000000001, 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.673 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94718 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.674 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685510+0000", 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685513+0000", 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4360000000000001e-06 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685519+0000", 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:13:36.675 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685523+0000", 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714918+0000", 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029395260999999999 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.676 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714985+0000", 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6737999999999997e-05 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715037+0000", 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1712999999999998e-05 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94719 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:36.677 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 8.9056663020000002, 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029657744, 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94719 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685529+0000", 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:13:36.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685535+0000", 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685539+0000", 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715062+0000", 2024-09-13T07:13:36.680 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029522738 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715125+0000", 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3229000000000001e-05 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715185+0000", 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9441999999999998e-05 2024-09-13T07:13:36.681 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:36.682 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:36.682 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:36.684 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:13:36.684 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:36.684 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:36.684+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:36.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:36.784 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:36.785+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:36.792 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:36.792 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:36.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:36.885 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:36.885+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:36.907 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:36.907 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:36.907 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:36.907 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:36.907 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:36.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:36.986 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:36.987+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:37.064 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51075 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529232346000001, 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081563839999999999, 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51075 2024-09-13T07:13:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535172+0000", 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535178+0000", 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535183+0000", 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000001e-06 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616726+0000", 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081543051000000005 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616734+0000", 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:13:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51076 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529215346000001, 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081579246999999994, 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51076 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535189+0000", 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9709999999999998e-06 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535195+0000", 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0739999999999998e-06 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535199+0000", 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5549999999999996e-06 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616759+0000", 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081559926000000005 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616766+0000", 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51077 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529198727999997, 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081595972000000003, 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51077 2024-09-13T07:13:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535203+0000", 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535210+0000", 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535217+0000", 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3490000000000003e-06 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535222+0000", 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616791+0000", 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081568733000000004 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616799+0000", 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.405e-06 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51078 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529175049999999, 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081603360999999999, 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51078 2024-09-13T07:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535229+0000", 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535236+0000", 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.489e-06 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535240+0000", 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616823+0000", 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081582882999999995 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616831+0000", 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2280000000000001e-06 2024-09-13T07:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51079 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529157597000001, 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081617985000000004, 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51079 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535244+0000", 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535247+0000", 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535253+0000", 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2249999999999997e-06 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535258+0000", 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8899999999999998e-06 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616855+0000", 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081597307999999993 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616863+0000", 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6580000000000002e-06 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51080 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529139665999999, 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081642305999999998, 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51080 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535262+0000", 2024-09-13T07:13:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535264+0000", 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535275+0000", 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0936e-05 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535281+0000", 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-13T07:13:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616889+0000", 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081608705000000004 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616905+0000", 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5398999999999999e-05 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51081 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529116678999998, 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081649656000000001, 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51081 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535285+0000", 2024-09-13T07:13:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535288+0000", 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535294+0000", 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535299+0000", 2024-09-13T07:13:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616928+0000", 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081629060000000003 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616935+0000", 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.968e-06 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51082 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529098668000003, 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081660766999999995, 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51082 2024-09-13T07:13:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535303+0000", 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535305+0000", 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535312+0000", 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535317+0000", 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616957+0000", 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081640502000000004 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616964+0000", 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51083 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529081034000001, 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081673832000000002, 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51083 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535323+0000", 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535329+0000", 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2650000000000002e-06 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535334+0000", 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616988+0000", 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081653901000000001 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616995+0000", 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1980000000000004e-06 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51084 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529063710999999, 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081686317999999994, 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51084 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535338+0000", 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535340+0000", 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0640000000000001e-06 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535347+0000", 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535351+0000", 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:13:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617017+0000", 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081665777999999994 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617025+0000", 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5789999999999998e-06 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51085 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529046329000003, 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081702650000000002, 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51085 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535358+0000", 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535364+0000", 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535369+0000", 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617051+0000", 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081681576000000006 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617059+0000", 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51086 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.529028535000002, 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081714234999999996, 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51086 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535373+0000", 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535376+0000", 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535382+0000", 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3899999999999998e-06 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535387+0000", 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7559999999999999e-06 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617081+0000", 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081694514999999995 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617088+0000", 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4439362310000003, 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82619132900000003, 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620465+0000", 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620483+0000", 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620497+0000", 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4248e-05 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620505+0000", 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9559999999999997e-06 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620563+0000", 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7667999999999997e-05 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620573+0000", 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9289999999999993e-06 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446642+0000", 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82606955599999998 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446657+0000", 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4932e-05 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4438706529999994, 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82648916299999997, 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620531+0000", 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620535+0000", 2024-09-13T07:13:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3959999999999999e-06 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620545+0000", 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5030000000000003e-06 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620551+0000", 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4690000000000002e-06 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620596+0000", 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4821000000000003e-05 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620604+0000", 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6359999999999994e-06 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447008+0000", 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82640484000000003 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447021+0000", 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2320999999999999e-05 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3501542569999998, 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73282608599999999, 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714247+0000", 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714303+0000", 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5475e-05 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714338+0000", 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5611000000000002e-05 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714345+0000", 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714410+0000", 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5223999999999998e-05 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714417+0000", 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447059+0000", 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73264227900000001 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447074+0000", 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5078e-05 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3499764869999993, 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73271206499999997, 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714425+0000", 2024-09-13T07:13:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714430+0000", 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714441+0000", 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0645999999999999e-05 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714450+0000", 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714489+0000", 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8974000000000003e-05 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714494+0000", 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447125+0000", 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73263175700000005 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447138+0000", 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2532e-05 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3448326299999991, 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72759390000000002, 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719569+0000", 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719586+0000", 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7397999999999998e-05 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719597+0000", 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0368e-05 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719603+0000", 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719649+0000", 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5818000000000001e-05 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719658+0000", 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0680000000000003e-06 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447157+0000", 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72749929599999996 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447164+0000", 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3659999999999997e-06 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3446304330000007, 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72744244800000002, 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719771+0000", 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719775+0000", 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719782+0000", 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3949999999999998e-06 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719789+0000", 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4719999999999998e-06 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719830+0000", 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1140000000000003e-05 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719841+0000", 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0808999999999999e-05 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447202+0000", 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72736066499999996 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447214+0000", 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2472e-05 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3313247500000003, 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71610967800000003, 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:37.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733095+0000", 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733107+0000", 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1902999999999999e-05 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733115+0000", 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733154+0000", 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8501000000000003e-05 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733159+0000", 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2449999999999998e-06 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447255+0000", 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71409564299999995 2024-09-13T07:13:37.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447366+0000", 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111273 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447510+0000", 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014420399999999999 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448696+0000", 2024-09-13T07:13:37.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001186246 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449068+0000", 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037139399999999999 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449145+0000", 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449187+0000", 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2407999999999998e-05 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [write 0~0] snapc 22=[22] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3311689680000001, 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71614144199999996, 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733232+0000", 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733237+0000", 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2880000000000003e-06 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733249+0000", 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2259000000000001e-05 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733256+0000", 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.742e-06 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733284+0000", 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8708e-05 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733294+0000", 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4089999999999992e-06 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447905+0000", 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71461088299999997 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448019+0000", 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011461800000000001 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448195+0000", 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017585699999999999 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448989+0000", 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079339899999999997 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449320+0000", 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000331825 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:37.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449350+0000", 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0111999999999999e-05 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449375+0000", 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4278e-05 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.152 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:37.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:37.154 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:37.154+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:37.254 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:37.255+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:37.301 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:37.302 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:37.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:37.354 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:37.355+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:37.417 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:37.417 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:37.417 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:37.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:37.469 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:37.470+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:37.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:13:37.545 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49523 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2699264~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.103226929999998, 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056285390999999997, 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49523 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442063+0000", 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9379999999999999e-06 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442068+0000", 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442073+0000", 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497684+0000", 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055610930000000003 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497759+0000", 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5199999999999998e-05 2024-09-13T07:13:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.498346+0000", 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058719100000000004 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49609 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 4075520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.085162599, 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055911012000000003, 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49609 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460127+0000", 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1430000000000001e-06 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460133+0000", 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5729999999999998e-06 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460138+0000", 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515783+0000", 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055645873999999998 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515861+0000", 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7979000000000007e-05 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.516036+0000", 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017504900000000001 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49618 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 999424~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.082648190999997, 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055816179, 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49618 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462639+0000", 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462642+0000", 2024-09-13T07:13:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462647+0000", 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462652+0000", 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518291+0000", 2024-09-13T07:13:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055639051000000002 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518368+0000", 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6913e-05 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518456+0000", 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8219000000000004e-05 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49619 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.082631765999999, 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055989537999999998, 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49619 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462659+0000", 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462665+0000", 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462669+0000", 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518483+0000", 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055813437 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518559+0000", 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5981000000000003e-05 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518646+0000", 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7175999999999993e-05 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49620 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1048576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.082613860999999, 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056152684000000001, 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49620 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462676+0000", 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462683+0000", 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462688+0000", 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518672+0000", 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055983988999999998 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518747+0000", 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4633000000000002e-05 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518827+0000", 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000002e-05 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49621 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1073152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.082595112999996, 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056323023999999999, 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49621 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462695+0000", 2024-09-13T07:13:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462701+0000", 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462706+0000", 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6530000000000002e-06 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518854+0000", 2024-09-13T07:13:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056147858000000002 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518934+0000", 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0813e-05 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519016+0000", 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1434999999999995e-05 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49622 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1097728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.082577827000001, 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056490088000000001, 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49622 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462712+0000", 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462718+0000", 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462722+0000", 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519042+0000", 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056320041000000001 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519120+0000", 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7607000000000004e-05 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519200+0000", 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166999999999993e-05 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49640 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1318912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.078773124000001, 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056480083, 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49640 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466514+0000", 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466517+0000", 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466524+0000", 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466529+0000", 2024-09-13T07:13:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520731+0000", 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054202469000000003 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520808+0000", 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6854000000000007e-05 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522995+0000", 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00218681 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49653 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1490944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.078651911999998, 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055956026999999998, 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49653 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466638+0000", 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466643+0000", 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466648+0000", 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522424+0000", 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055775681000000001 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522499+0000", 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5828000000000007e-05 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522592+0000", 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2781000000000005e-05 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49674 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1662976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.077238522999998, 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056005177000000003, 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49674 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468052+0000", 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.497e-06 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468058+0000", 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2430000000000002e-06 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468063+0000", 2024-09-13T07:13:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523872+0000", 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055808706999999999 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523960+0000", 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8919999999999996e-05 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.524055+0000", 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4222999999999996e-05 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49695 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.074952862000004, 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056046466000000003, 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49695 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470338+0000", 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470345+0000", 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4100000000000002e-06 2024-09-13T07:13:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470351+0000", 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525756+0000", 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055404954999999999 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525819+0000", 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2779000000000003e-05 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526382+0000", 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056291800000000003 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49696 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:37.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.074932164000003, 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056222355000000002, 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49696 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470355+0000", 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470358+0000", 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470364+0000", 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470369+0000", 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526411+0000", 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056041476999999999 2024-09-13T07:13:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526489+0000", 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7787000000000006e-05 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526578+0000", 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9606999999999998e-05 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49704 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2056192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.073306313000003, 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055974318000000002, 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49704 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471981+0000", 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471984+0000", 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471990+0000", 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7229999999999998e-06 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471995+0000", 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527788+0000", 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055793091000000003 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527861+0000", 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3038000000000001e-05 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527956+0000", 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4536000000000004e-05 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49706 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2080768~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.073288505999997, 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056135461999999997, 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49706 2024-09-13T07:13:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472006+0000", 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472012+0000", 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472017+0000", 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527980+0000", 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055963185999999998 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528050+0000", 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979000000000002e-05 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528135+0000", 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4594999999999999e-05 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49708 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2105344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.073266345, 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056284837999999997, 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49708 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472021+0000", 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472024+0000", 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.261e-06 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472029+0000", 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472034+0000", 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0069999999999997e-06 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528161+0000", 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056126285999999997 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528232+0000", 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1000000000000005e-05 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528306+0000", 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4876000000000006e-05 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49711 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2129920~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.073248831000001, 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444877999999997, 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49711 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472041+0000", 2024-09-13T07:13:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472047+0000", 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472052+0000", 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528331+0000", 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056278504 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528399+0000", 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7941000000000006e-05 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528484+0000", 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5249999999999999e-05 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49713 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2154496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.073077517999998, 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444079000000001, 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49713 2024-09-13T07:13:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472214+0000", 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472228+0000", 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3964999999999999e-05 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472236+0000", 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4109999999999999e-06 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528509+0000", 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056273532000000001 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528581+0000", 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1338e-05 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528655+0000", 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3858999999999999e-05 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49728 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2326528~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.071203617000002, 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055971704999999997, 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49728 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474087+0000", 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474093+0000", 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T07:13:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474098+0000", 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529880+0000", 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055782102 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529967+0000", 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6853000000000003e-05 2024-09-13T07:13:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.530056+0000", 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9430000000000003e-05 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49767 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2695168~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.067113941000002, 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062028049000000002, 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49767 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478176+0000", 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478182+0000", 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478187+0000", 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532704+0000", 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054517200000000002 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532779+0000", 2024-09-13T07:13:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5244e-05 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.540202+0000", 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074227429999999999 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:50078 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.053311761000003, 2024-09-13T07:13:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055871918999999999, 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50078 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491979+0000", 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491986+0000", 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491991+0000", 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544160+0000", 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052168953999999997 2024-09-13T07:13:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544241+0000", 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1213999999999999e-05 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.547848+0000", 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003606663 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:37.633 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:37.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:37.634 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:37.635+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:37.735 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:37.736+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:37.781 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:37.781 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:37.781 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:37.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:37.836 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:37.837+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:37.936 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:37.937 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:37.937 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:37.938+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:37.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:38.037 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:38.039+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:38.053 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.755842821000002, 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.754364795999997, 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.751547747000004, 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.182118787, 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.169495191000003, 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.425913422999997, 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3913 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'56, mlcod=29'53)", 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.425376196000002, 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029359099999999999, 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628388+0000", 2024-09-13T07:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628392+0000", 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839999999999998e-06 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628401+0000", 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6500000000000002e-06 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628406+0000", 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628466+0000", 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9483e-05 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.628479+0000", 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3285000000000001e-05 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631293+0000", 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028141709999999999 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.631324+0000", 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1111000000000001e-05 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.286294560000002, 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.238947267, 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.211463162000001, 2024-09-13T07:13:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:13:38.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.205448738000001, 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:38.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4245 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'137, mlcod=29'132)", 2024-09-13T07:13:38.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.153756444000003, 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002919624, 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900008+0000", 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900013+0000", 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900020+0000", 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900023+0000", 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900060+0000", 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6452999999999998e-05 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.900077+0000", 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7725999999999999e-05 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902892+0000", 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028143389999999999 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.902928+0000", 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5966000000000001e-05 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.617532595999997, 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:38.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:38.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.857379418000001, 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:13:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:13:38.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.346970728999999, 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:38.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:64537 3.9 e47/19 3:97a74db9:::rbd_data.117332306d66.0000000000000002:head v 47'372, mlcod=45'370)", 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.434357296999998, 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092884539999999998, 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.624988+0000", 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055812279999999997 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625018+0000", 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9496999999999999e-05 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625031+0000", 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625078+0000", 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7290000000000003e-05 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625096+0000", 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8091999999999999e-05 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628662+0000", 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035652449999999999 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628696+0000", 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4035999999999997e-05 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:71538 3.0 e50/19 3:014540de:::rbd_data.1173f6918137.0000000000000082:head v 50'331, mlcod=49'326)", 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.635209597999999, 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032256870000000001, 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418554+0000", 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418562+0000", 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4000000000000003e-06 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418573+0000", 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1685000000000001e-05 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418580+0000", 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9419999999999998e-06 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418611+0000", 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1053000000000002e-05 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418630+0000", 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421749+0000", 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031190620000000001 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421781+0000", 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1905e-05 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:85070 3.0 e58/19 3:077f600c:::rbd_data.11732319a2dc.0000000000000001:head v 58'356, mlcod=57'354)", 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.437944870999999, 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029694840000000001, 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615818+0000", 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616402+0000", 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058395300000000001 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616417+0000", 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4630000000000001e-05 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616424+0000", 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616469+0000", 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4508000000000001e-05 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616499+0000", 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0491999999999999e-05 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618737+0000", 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002238037 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618789+0000", 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2055999999999998e-05 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:86269 3.9 e58/19 3:96e8c531:::rbd_data.11732319a2dc.00000000000000bd:head v 58'427, mlcod=58'425)", 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.865781102, 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030219940000000001, 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187992+0000", 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1339999999999993e-06 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188006+0000", 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939e-05 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188013+0000", 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188067+0000", 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4190000000000001e-05 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188085+0000", 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309e-05 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.190969+0000", 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028840379999999998 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.191005+0000", 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6319999999999998e-05 2024-09-13T07:13:38.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:97494 3.0 e67/19 3:02ad2de2:::rbd_data.117387780896.0000000000000002:head v 67'375, mlcod=66'372)", 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1801559239999992, 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005628437, 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873608+0000", 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875823+0000", 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022143660000000002 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875843+0000", 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9864000000000002e-05 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875854+0000", 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1345999999999999e-05 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875918+0000", 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4151000000000003e-05 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875939+0000", 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1016999999999999e-05 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879194+0000", 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032546820000000001 2024-09-13T07:13:38.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879237+0000", 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3353000000000003e-05 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.141 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:38.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:38.143 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:38.143+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:38.243 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:38.244+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:38.291 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:38.291 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:38.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:38.383 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:38.384+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:38.438 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:38.444 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:38.446+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:38.446 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:38.446 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:38.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:38.511 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:13:38.545 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:38.546+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:38.579 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:38.608 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:38.608 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:38.641 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:38.641 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084661799000003, 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:13:38.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084636484999997, 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:38.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084602119000003, 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:13:38.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084573294999998, 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:13:38.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084554998000002, 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:13:38.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:13:38.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084530192000003, 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:13:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:13:38.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084512318999998, 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:13:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:13:38.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084485977999996, 2024-09-13T07:13:38.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:13:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:13:38.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084459256000002, 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084441671, 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084415032999999, 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:13:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084397572999997, 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:13:38.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084370614999997, 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.706 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.706 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:38.706 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084344410999996, 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:13:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:13:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084326419, 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:13:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084234651999999, 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:13:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084212352000002, 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:13:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084181350999998, 2024-09-13T07:13:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:13:38.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084156595000003, 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:38.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.084138903000003, 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:13:38.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:38.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:38.740 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:38.741 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:38.646+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:38.742 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:38.742 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:38.742 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:38.743 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:38.743 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:38.743 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:38.744 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:38.744 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:38.744 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:38.745 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:38.745 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:38.747 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:38.747+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:38.847 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:38.848+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:38.848 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:38.848 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:38.857 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:38.948 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:38.949+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:38.959 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:38.959 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:38.967 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:39.049 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:39.050+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:39.110 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:39.110 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:39.118 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:39.150 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:39.151+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:39.220 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:39.220 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:39.228 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:39.273 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:39.274+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:39.331 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:39.331 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:39.340 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:39.351 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:39.353+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:39.452 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:39.453+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:39.483 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:39.483 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:39.492 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:39.577 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:39.578+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:39.646 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:39.646 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:39.653 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:39.654+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:39.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:39.754 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:39.755+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:39.797 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:39.797 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:39.806 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:39.855 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:39.856+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:39.908 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:39.908 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:39.916 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:39.960 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:39.961+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:40.018 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:40.019 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:40.028 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:40.069 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:40.070+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:40.158 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:40.159+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:40.170 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:40.171 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:40.179 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:40.259 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:40.260+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:40.322 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:40.322 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:40.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:40.374 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:40.375+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:40.449 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:40.450 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:40.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:40.461 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:40.463+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:40.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:13:40.562 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:40.563+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:40.622 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:40.622 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:40.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:40.677 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:40.678+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:40.744 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:40.744 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:40.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:40.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:40.794 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:40.795+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:40.864 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:40.865+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:40.873 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:40.873 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:40.905 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:40.906 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:40.917 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:40.918 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:40.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:40.922 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:40.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:40.938 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:40.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:40.965 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:40.966+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:40.972 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:40.972 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:40.989 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:40.989 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:41.005 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:41.005 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:41.034 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:41.034 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:41.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:41.084 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:41.085+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:41.166 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:41.167+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:41.191 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49523 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2699264~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.749574293999999, 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056285390999999997, 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49523 2024-09-13T07:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442063+0000", 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9379999999999999e-06 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442068+0000", 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442073+0000", 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497684+0000", 2024-09-13T07:13:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055610930000000003 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497759+0000", 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5199999999999998e-05 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.498346+0000", 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058719100000000004 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49609 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 4075520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.731509963000001, 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055911012000000003, 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49609 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460127+0000", 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1430000000000001e-06 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460133+0000", 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5729999999999998e-06 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460138+0000", 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515783+0000", 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055645873999999998 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515861+0000", 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7979000000000007e-05 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.516036+0000", 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017504900000000001 2024-09-13T07:13:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49618 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 999424~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.728995554999997, 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055816179, 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49618 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462639+0000", 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462642+0000", 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462647+0000", 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462652+0000", 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518291+0000", 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055639051000000002 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518368+0000", 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6913e-05 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518456+0000", 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8219000000000004e-05 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49619 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.728979129999999, 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055989537999999998, 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49619 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462659+0000", 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462665+0000", 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462669+0000", 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518483+0000", 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055813437 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518559+0000", 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5981000000000003e-05 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518646+0000", 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7175999999999993e-05 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49620 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1048576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.728961224999999, 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056152684000000001, 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49620 2024-09-13T07:13:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462676+0000", 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462683+0000", 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462688+0000", 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518672+0000", 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055983988999999998 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518747+0000", 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4633000000000002e-05 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518827+0000", 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000002e-05 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49621 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1073152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.728942476999997, 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056323023999999999, 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49621 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462695+0000", 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462701+0000", 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462706+0000", 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6530000000000002e-06 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518854+0000", 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056147858000000002 2024-09-13T07:13:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518934+0000", 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0813e-05 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519016+0000", 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1434999999999995e-05 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49622 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1097728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.728925191000002, 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056490088000000001, 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49622 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462712+0000", 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462718+0000", 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462722+0000", 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:13:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519042+0000", 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056320041000000001 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519120+0000", 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7607000000000004e-05 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519200+0000", 2024-09-13T07:13:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166999999999993e-05 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49640 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1318912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.725120488000002, 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056480083, 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49640 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466514+0000", 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466517+0000", 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466524+0000", 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T07:13:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466529+0000", 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520731+0000", 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054202469000000003 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520808+0000", 2024-09-13T07:13:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6854000000000007e-05 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522995+0000", 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00218681 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49653 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1490944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.724999275999998, 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055956026999999998, 2024-09-13T07:13:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49653 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466638+0000", 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-13T07:13:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466643+0000", 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466648+0000", 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522424+0000", 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055775681000000001 2024-09-13T07:13:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522499+0000", 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5828000000000007e-05 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522592+0000", 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2781000000000005e-05 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49674 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1662976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.723585886999999, 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056005177000000003, 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49674 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468052+0000", 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.497e-06 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468058+0000", 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2430000000000002e-06 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468063+0000", 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T07:13:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523872+0000", 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055808706999999999 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523960+0000", 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8919999999999996e-05 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.524055+0000", 2024-09-13T07:13:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4222999999999996e-05 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49695 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.721300225999997, 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056046466000000003, 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49695 2024-09-13T07:13:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470338+0000", 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470345+0000", 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4100000000000002e-06 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470351+0000", 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525756+0000", 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055404954999999999 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525819+0000", 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2779000000000003e-05 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526382+0000", 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056291800000000003 2024-09-13T07:13:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49696 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.721279527999997, 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056222355000000002, 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49696 2024-09-13T07:13:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470355+0000", 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470358+0000", 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470364+0000", 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470369+0000", 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526411+0000", 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056041476999999999 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526489+0000", 2024-09-13T07:13:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7787000000000006e-05 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526578+0000", 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9606999999999998e-05 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49704 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2056192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.719653676999997, 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055974318000000002, 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49704 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471981+0000", 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471984+0000", 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471990+0000", 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7229999999999998e-06 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471995+0000", 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527788+0000", 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055793091000000003 2024-09-13T07:13:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527861+0000", 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3038000000000001e-05 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527956+0000", 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4536000000000004e-05 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49706 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2080768~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.719635869999998, 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056135461999999997, 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49706 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472006+0000", 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472012+0000", 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472017+0000", 2024-09-13T07:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527980+0000", 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055963185999999998 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528050+0000", 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979000000000002e-05 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528135+0000", 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4594999999999999e-05 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49708 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2105344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.719613709000001, 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056284837999999997, 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49708 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472021+0000", 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472024+0000", 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.261e-06 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472029+0000", 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472034+0000", 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0069999999999997e-06 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528161+0000", 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056126285999999997 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528232+0000", 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1000000000000005e-05 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528306+0000", 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4876000000000006e-05 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49711 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2129920~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.719596195000001, 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444877999999997, 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49711 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472041+0000", 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472047+0000", 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472052+0000", 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-09-13T07:13:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528331+0000", 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056278504 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528399+0000", 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7941000000000006e-05 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528484+0000", 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5249999999999999e-05 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49713 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2154496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.719424881999998, 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444079000000001, 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49713 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472214+0000", 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472228+0000", 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3964999999999999e-05 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472236+0000", 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4109999999999999e-06 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528509+0000", 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056273532000000001 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528581+0000", 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1338e-05 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528655+0000", 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3858999999999999e-05 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49728 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2326528~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.717550981000002, 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055971704999999997, 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49728 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474087+0000", 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474093+0000", 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474098+0000", 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529880+0000", 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055782102 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529967+0000", 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6853000000000003e-05 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.530056+0000", 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9430000000000003e-05 2024-09-13T07:13:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49767 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2695168~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.713461305000003, 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062028049000000002, 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49767 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478176+0000", 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478182+0000", 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T07:13:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478187+0000", 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532704+0000", 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054517200000000002 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532779+0000", 2024-09-13T07:13:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5244e-05 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.540202+0000", 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074227429999999999 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:50078 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.699659124999997, 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055871918999999999, 2024-09-13T07:13:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50078 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491979+0000", 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-13T07:13:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491986+0000", 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491991+0000", 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544160+0000", 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052168953999999997 2024-09-13T07:13:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544241+0000", 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1213999999999999e-05 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.547848+0000", 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003606663 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.280 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:41.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:41.281 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:41.282+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:41.382 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:41.383+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:41.428 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:41.429 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:41.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:41.483 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:41.484+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:41.518 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:13:41.553 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:41.553 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:41.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:41.603 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:41.604+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:41.685 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:41.685 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:41.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:13:41.686 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:41.686+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:41.713 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.415812516000003, 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.414334491000005, 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.411517442000005, 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.842088482000001, 2024-09-13T07:13:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3066 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'107, mlcod=27'105)", 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.829464885999997, 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032103570000000001, 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884269+0000", 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884268+0000", 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884277+0000", 2024-09-13T07:13:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884294+0000", 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6218999999999998e-05 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884300+0000", 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T07:13:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884343+0000", 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3534e-05 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.884364+0000", 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0302999999999999e-05 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887442+0000", 2024-09-13T07:13:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030785729999999998 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.887480+0000", 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7509000000000003e-05 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3912 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'55, mlcod=29'53)", 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.085883117999998, 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031311960000000002, 2024-09-13T07:13:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627851+0000", 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627856+0000", 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627866+0000", 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0293e-05 2024-09-13T07:13:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627880+0000", 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4354e-05 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627933+0000", 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769000000000002e-05 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.627947+0000", 2024-09-13T07:13:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3791000000000001e-05 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630944+0000", 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029965790000000001 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.630982+0000", 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.946264255000003, 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.898916962000001, 2024-09-13T07:13:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4191 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'120, mlcod=29'117)", 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.871432857000002, 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030084949999999999, 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842301+0000", 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842307+0000", 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7509999999999997e-06 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842318+0000", 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1453e-05 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842324+0000", 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842361+0000", 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6968000000000002e-05 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.842373+0000", 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1875000000000001e-05 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845275+0000", 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002902253 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.845310+0000", 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4612e-05 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.865418433000002, 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.277502290999998, 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:8471 3.0 e34/19 3:0547d48a:::rbd_data.117344605995.0000000000000017:head v 34'229, mlcod=34'226)", 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.517349113000002, 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031493210000000001, 2024-09-13T07:13:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196385+0000", 2024-09-13T07:13:41.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196390+0000", 2024-09-13T07:13:41.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:13:41.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196399+0000", 2024-09-13T07:13:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2650000000000002e-06 2024-09-13T07:13:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196403+0000", 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196431+0000", 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7469000000000001e-05 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.196443+0000", 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2181000000000001e-05 2024-09-13T07:13:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199497+0000", 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003054771 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:50.199534+0000", 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6987999999999999e-05 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.006940424, 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:64537 3.9 e47/19 3:97a74db9:::rbd_data.117332306d66.0000000000000002:head v 47'372, mlcod=45'370)", 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.094326991999999, 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092884539999999998, 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.624988+0000", 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055812279999999997 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625018+0000", 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9496999999999999e-05 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625031+0000", 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625078+0000", 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7290000000000003e-05 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625096+0000", 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8091999999999999e-05 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628662+0000", 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035652449999999999 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628696+0000", 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4035999999999997e-05 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:71538 3.0 e50/19 3:014540de:::rbd_data.1173f6918137.0000000000000082:head v 50'331, mlcod=49'326)", 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.295179292999997, 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032256870000000001, 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418555+0000", 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418554+0000", 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418562+0000", 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4000000000000003e-06 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418573+0000", 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1685000000000001e-05 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418580+0000", 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9419999999999998e-06 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418611+0000", 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1053000000000002e-05 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.418630+0000", 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421749+0000", 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031190620000000001 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:09.421781+0000", 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1905e-05 2024-09-13T07:13:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:85070 3.0 e58/19 3:077f600c:::rbd_data.11732319a2dc.0000000000000001:head v 58'356, mlcod=57'354)", 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.097914566, 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029694840000000001, 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615820+0000", 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.615818+0000", 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616402+0000", 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058395300000000001 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616417+0000", 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4630000000000001e-05 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616424+0000", 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5209999999999997e-06 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616469+0000", 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4508000000000001e-05 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.616499+0000", 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0491999999999999e-05 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618737+0000", 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002238037 2024-09-13T07:13:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:18.618789+0000", 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2055999999999998e-05 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:86269 3.9 e58/19 3:96e8c531:::rbd_data.11732319a2dc.00000000000000bd:head v 58'427, mlcod=58'425)", 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.525750797000001, 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030219940000000001, 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187983+0000", 2024-09-13T07:13:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.187992+0000", 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1339999999999993e-06 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188006+0000", 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939e-05 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188013+0000", 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188067+0000", 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4190000000000001e-05 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.188085+0000", 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309e-05 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.190969+0000", 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028840379999999998 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:20.191005+0000", 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6319999999999998e-05 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:97494 3.0 e67/19 3:02ad2de2:::rbd_data.117387780896.0000000000000002:head v 67'375, mlcod=66'372)", 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.840125619, 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005628437, 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873608+0000", 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875823+0000", 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022143660000000002 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875843+0000", 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9864000000000002e-05 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875854+0000", 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1345999999999999e-05 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875918+0000", 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4151000000000003e-05 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875939+0000", 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1016999999999999e-05 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879194+0000", 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032546820000000001 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879237+0000", 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3353000000000003e-05 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1144 3.4 e76/19 3:21ae49f8:::rbd_data.11738a001609.000000000000000c:22 v 76'476, mlcod=71'472)", 2024-09-13T07:13:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:38.515557+0000", 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1981767670000001, 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032617810000000001, 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515557+0000", 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515557+0000", 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515557+0000", 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515565+0000", 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5390000000000002e-06 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515576+0000", 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1382999999999999e-05 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515593+0000", 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6871000000000002e-05 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515711+0000", 2024-09-13T07:13:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011854900000000001 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515733+0000", 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1183e-05 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.518772+0000", 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030398859999999999 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.518819+0000", 2024-09-13T07:13:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6866000000000002e-05 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1145 3.4 e76/19 3:27c86583:::rbd_data.11738a001609.0000000000000046:22 v 76'478, mlcod=71'472)", 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1981269590000001, 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063983709999999999, 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515618+0000", 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0787e-05 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515629+0000", 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0905e-05 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515634+0000", 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1710000000000001e-06 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.516333+0000", 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00069910600000000001 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.516351+0000", 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7822999999999999e-05 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.521968+0000", 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056176519999999999 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.522006+0000", 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7246000000000002e-05 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:41.805 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:41.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:41.807 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:41.807+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:41.907 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:41.908+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:41.954 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:41.956 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:41.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:13:41.974 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:13:41.984 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-13T07:13:41.518682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":936703150,"num_objects":1570,"num_object_clones":356,"num_object_copies":3140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1570,"num_whiteouts":753,"num_read":140468,"num_read_kb":621176,"num_write":18585,"num_write_kb":358605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14524,"ondisk_log_size":14524,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383032,"kb_used_data":49324,"kb_used_omap":20,"kb_used_meta":333291,"kb_avail":754591688,"statfs":{"total":773094113280,"available":772701888512,"internally_reserved":0,"allocated":50507776,"data_stored":366059167,"data_compressed":10146410,"data_compressed_allocated":27512832,"data_compressed_original":348779593,"omap_allocated":21248,"internal_metadata":341290240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-91283622,"num_objects":-195,"num_object_clones":-71,"num_object_copies":-390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-195,"num_whiteouts":-33,"num_read":942,"num_read_kb":809,"num_write":234,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019536"},"pg_stats":[{"pgid":"3.1f","version":"76'311","reported_seq":3848,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.631064+0000","last_change":"2024-09-13T07:13:38.535048+0000","last_active":"2024-09-13T07:13:38.631064+0000","last_peered":"2024-09-13T07:13:38.631064+0000","last_clean":"2024-09-13T07:13:38.631064+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:13:38.631064+0000","last_undegraded":"2024-09-13T07:13:38.631064+0000","last_fullsized":"2024-09-13T07:13:38.631064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.023669808000000001,"stat_sum":{"num_bytes":29114121,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":3978,"num_read_kb":17485,"num_write":342,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1e","version":"76'321","reported_seq":4185,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.607072+0000","last_change":"2024-09-13T07:13:38.538024+0000","last_active":"2024-09-13T07:13:38.607072+0000","last_peered":"2024-09-13T07:13:38.607072+0000","last_clean":"2024-09-13T07:13:38.607072+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:13:38.607072+0000","last_undegraded":"2024-09-13T07:13:38.607072+0000","last_fullsized":"2024-09-13T07:13:38.607072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.026289139999999999,"stat_sum":{"num_bytes":30927693,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":40,"num_read":4589,"num_read_kb":20234,"num_write":279,"num_write_kb":5817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1d","version":"76'569","reported_seq":2349,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.599797+0000","last_change":"2024-09-13T07:13:38.513921+0000","last_active":"2024-09-13T07:13:38.599797+0000","last_peered":"2024-09-13T07:13:38.599797+0000","last_clean":"2024-09-13T07:13:38.599797+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:13:38.599797+0000","last_undegraded":"2024-09-13T07:13:38.599797+0000","last_fullsized":"2024-09-13T07:13:38.599797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0066794910000000001,"stat_sum":{"num_bytes":28123595,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":27,"num_read":4066,"num_read_kb":14782,"num_write":829,"num_write_kb":8713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1c","version":"76'449","reported_seq":3311,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.625716+0000","last_change":"2024-09-13T07:13:38.509709+0000","last_active":"2024-09-13T07:13:38.625716+0000","last_peered":"2024-09-13T07:13:38.625716+0000","last_clean":"2024-09-13T07:13:38.625716+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:13:38.625716+0000","last_undegraded":"2024-09-13T07:13:38.625716+0000","last_fullsized":"2024-09-13T07:13:38.625716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.002645194,"stat_sum":{"num_bytes":29323388,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":3856,"num_read_kb":23164,"num_write":568,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1b","version":"76'1022","reported_seq":4386,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.598879+0000","last_change":"2024-09-13T07:13:38.537516+0000","last_active":"2024-09-13T07:13:38.598879+0000","last_peered":"2024-09-13T07:13:38.598879+0000","last_clean":"2024-09-13T07:13:38.598879+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:13:38.598879+0000","last_undegraded":"2024-09-13T07:13:38.598879+0000","last_fullsized":"2024-09-13T07:13:38.598879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.026137489,"stat_sum":{"num_bytes":37703333,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":7873,"num_read_kb":35463,"num_write":1477,"num_write_kb":7819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1a","version":"76'300","reported_seq":3906,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.643004+0000","last_change":"2024-09-13T07:13:38.517915+0000","last_active":"2024-09-13T07:13:38.643004+0000","last_peered":"2024-09-13T07:13:38.643004+0000","last_clean":"2024-09-13T07:13:38.643004+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:13:38.643004+0000","last_undegraded":"2024-09-13T07:13:38.643004+0000","last_fullsized":"2024-09-13T07:13:38.643004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0080293459999999997,"stat_sum":{"num_bytes":23734719,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":4089,"num_read_kb":22663,"num_write":287,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.19","version":"76'303","reported_seq":5079,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.606492+0000","last_change":"2024-09-13T07:13:38.527177+0000","last_active":"2024-09-13T07:13:38.606492+0000","last_peered":"2024-09-13T07:13:38.606492+0000","last_clean":"2024-09-13T07:13:38.606492+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:13:38.606492+0000","last_undegraded":"2024-09-13T07:13:38.606492+0000","last_fullsized":"2024-09-13T07:13:38.606492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.018179284,"stat_sum":{"num_bytes":31410876,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":5001,"num_read_kb":22900,"num_write":341,"num_write_kb":19603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.18","version":"76'458","reported_seq":5045,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.596866+0000","last_change":"2024-09-13T07:13:38.515676+0000","last_active":"2024-09-13T07:13:38.596866+0000","last_peered":"2024-09-13T07:13:38.596866+0000","last_clean":"2024-09-13T07:13:38.596866+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:13:38.596866+0000","last_undegraded":"2024-09-13T07:13:38.596866+0000","last_fullsized":"2024-09-13T07:13:38.596866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057951210000000003,"stat_sum":{"num_bytes":33050874,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":34,"num_read":7007,"num_read_kb":28476,"num_write":516,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.17","version":"76'433","reported_seq":2804,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.601597+0000","last_change":"2024-09-13T07:13:38.533038+0000","last_active":"2024-09-13T07:13:38.601597+0000","last_peered":"2024-09-13T07:13:38.601597+0000","last_clean":"2024-09-13T07:13:38.601597+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:13:38.601597+0000","last_undegraded":"2024-09-13T07:13:38.601597+0000","last_fullsized":"2024-09-13T07:13:38.601597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.022033893999999998,"stat_sum":{"num_bytes":23206615,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":4080,"num_read_kb":16378,"num_write":439,"num_write_kb":7155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.16","version":"76'222","reported_seq":1173,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.510188+0000","last_change":"2024-09-13T07:13:38.510188+0000","last_active":"2024-09-13T07:13:38.510188+0000","last_peered":"2024-09-13T07:13:38.510188+0000","last_clean":"2024-09-13T07:13:38.510188+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:13:38.510188+0000","last_undegraded":"2024-09-13T07:13:38.510188+0000","last_fullsized":"2024-09-13T07:13:38.510188+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030779449999999999,"stat_sum":{"num_bytes":24453565,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":28,"num_read":843,"num_read_kb":7353,"num_write":210,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.15","version":"74'298","reported_seq":4444,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T07:13:36.575438+0000","last_change":"2024-09-13T07:13:36.575438+0000","last_active":"2024-09-13T07:13:36.575438+0000","last_peered":"2024-09-13T07:13:36.575438+0000","last_clean":"2024-09-13T07:13:36.575438+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:13:36.575438+0000","last_undegraded":"2024-09-13T07:13:36.575438+0000","last_fullsized":"2024-09-13T07:13:36.575438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092579000000000001,"stat_sum":{"num_bytes":21497076,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":4232,"num_read_kb":14680,"num_write":380,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"3"}]},{"pgid":"3.14","version":"76'336","reported_seq":1784,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.520264+0000","last_change":"2024-09-13T07:13:38.520264+0000","last_active":"2024-09-13T07:13:38.520264+0000","last_peered":"2024-09-13T07:13:38.520264+0000","last_clean":"2024-09-13T07:13:38.520264+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:13:38.520264+0000","last_undegraded":"2024-09-13T07:13:38.520264+0000","last_fullsized":"2024-09-13T07:13:38.520264+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010179817000000001,"stat_sum":{"num_bytes":40356097,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":2300,"num_read_kb":13598,"num_write":399,"num_write_kb":4127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T07:13:37.504363+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:13:37.504363+0000","last_peered":"2024-09-13T07:13:37.504363+0000","last_clean":"2024-09-13T07:13:37.504363+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:13:37.504363+0000","last_undegraded":"2024-09-13T07:13:37.504363+0000","last_fullsized":"2024-09-13T07:13:37.504363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"76'379","reported_seq":3497,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.612440+0000","last_change":"2024-09-13T07:13:38.525470+0000","last_active":"2024-09-13T07:13:38.612440+0000","last_peered":"2024-09-13T07:13:38.612440+0000","last_clean":"2024-09-13T07:13:38.612440+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:13:38.612440+0000","last_undegraded":"2024-09-13T07:13:38.612440+0000","last_fullsized":"2024-09-13T07:13:38.612440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018300219999999999,"stat_sum":{"num_bytes":13315710,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":4662,"num_read_kb":15996,"num_write":438,"num_write_kb":4945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T07:13:37.504493+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:13:37.504493+0000","last_peered":"2024-09-13T07:13:37.504493+0000","last_clean":"2024-09-13T07:13:37.504493+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:13:37.504493+0000","last_undegraded":"2024-09-13T07:13:37.504493+0000","last_fullsized":"2024-09-13T07:13:37.504493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"76'481","reported_seq":4054,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.605864+0000","last_change":"2024-09-13T07:13:38.523284+0000","last_active":"2024-09-13T07:13:38.605864+0000","last_peered":"2024-09-13T07:13:38.605864+0000","last_clean":"2024-09-13T07:13:38.605864+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:13:38.605864+0000","last_undegraded":"2024-09-13T07:13:38.605864+0000","last_fullsized":"2024-09-13T07:13:38.605864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013908098000000001,"stat_sum":{"num_bytes":20635380,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":5087,"num_read_kb":13978,"num_write":612,"num_write_kb":8116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T07:13:37.501092+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:13:37.501092+0000","last_peered":"2024-09-13T07:13:37.501092+0000","last_clean":"2024-09-13T07:13:37.501092+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:13:37.501092+0000","last_undegraded":"2024-09-13T07:13:37.501092+0000","last_fullsized":"2024-09-13T07:13:37.501092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"76'342","reported_seq":773,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.657192+0000","last_change":"2024-09-13T07:13:38.512842+0000","last_active":"2024-09-13T07:13:38.657192+0000","last_peered":"2024-09-13T07:13:38.657192+0000","last_clean":"2024-09-13T07:13:38.657192+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:13:38.657192+0000","last_undegraded":"2024-09-13T07:13:38.657192+0000","last_fullsized":"2024-09-13T07:13:38.657192+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025536669999999999,"stat_sum":{"num_bytes":8819496,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":1136,"num_read_kb":5351,"num_write":445,"num_write_kb":2915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T07:13:37.501728+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:13:37.501728+0000","last_peered":"2024-09-13T07:13:37.501728+0000","last_clean":"2024-09-13T07:13:37.501728+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:13:37.501728+0000","last_undegraded":"2024-09-13T07:13:37.501728+0000","last_fullsized":"2024-09-13T07:13:37.501728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"76'311","reported_seq":7595,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.616872+0000","last_change":"2024-09-13T07:13:38.524628+0000","last_active":"2024-09-13T07:13:38.616872+0000","last_peered":"2024-09-13T07:13:38.616872+0000","last_clean":"2024-09-13T07:13:38.616872+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:13:38.616872+0000","last_undegraded":"2024-09-13T07:13:38.616872+0000","last_fullsized":"2024-09-13T07:13:38.616872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015034831,"stat_sum":{"num_bytes":16874429,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":7463,"num_read_kb":20704,"num_write":348,"num_write_kb":18762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.1","version":"76'508","reported_seq":4432,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.600464+0000","last_change":"2024-09-13T07:13:38.527143+0000","last_active":"2024-09-13T07:13:38.600464+0000","last_peered":"2024-09-13T07:13:38.600464+0000","last_clean":"2024-09-13T07:13:38.600464+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:13:38.600464+0000","last_undegraded":"2024-09-13T07:13:38.600464+0000","last_fullsized":"2024-09-13T07:13:38.600464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.017860816000000002,"stat_sum":{"num_bytes":12541552,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":5341,"num_read_kb":29525,"num_write":752,"num_write_kb":11453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"2.0","version":"17'117","reported_seq":191,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T07:13:35.555370+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:13:35.555370+0000","last_peered":"2024-09-13T07:13:35.555370+0000","last_clean":"2024-09-13T07:13:35.555370+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:13:35.555370+0000","last_undegraded":"2024-09-13T07:13:35.555370+0000","last_fullsized":"2024-09-13T07:13:35.555370+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":76,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T07:13:37.500127+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:13:37.500127+0000","last_peered":"2024-09-13T07:13:37.500127+0000","last_clean":"2024-09-13T07:13:37.500127+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:13:37.500127+0000","last_undegraded":"2024-09-13T07:13:37.500127+0000","last_fullsized":"2024-09-13T07:13:37.500127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"76'414","reported_seq":2469,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.595501+0000","last_change":"2024-09-13T07:13:38.527980+0000","last_active":"2024-09-13T07:13:38.595501+0000","last_peered":"2024-09-13T07:13:38.595501+0000","last_clean":"2024-09-13T07:13:38.595501+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:13:38.595501+0000","last_undegraded":"2024-09-13T07:13:38.595501+0000","last_fullsized":"2024-09-13T07:13:38.595501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.018552035000000001,"stat_sum":{"num_bytes":38176098,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":3079,"num_read_kb":26596,"num_write":608,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.2","version":"17'2","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T07:13:35.555442+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:13:35.555442+0000","last_peered":"2024-09-13T07:13:35.555442+0000","last_clean":"2024-09-13T07:13:35.555442+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:13:35.555442+0000","last_undegraded":"2024-09-13T07:13:35.555442+0000","last_fullsized":"2024-09-13T07:13:35.555442+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'437","reported_seq":3951,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.635920+0000","last_change":"2024-09-13T07:13:38.517806+0000","last_active":"2024-09-13T07:13:38.635920+0000","last_peered":"2024-09-13T07:13:38.635920+0000","last_clean":"2024-09-13T07:13:38.635920+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:13:38.635920+0000","last_undegraded":"2024-09-13T07:13:38.635920+0000","last_fullsized":"2024-09-13T07:13:38.635920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0078248510000000007,"stat_sum":{"num_bytes":49253623,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":35,"num_read":4610,"num_read_kb":23384,"num_write":547,"num_write_kb":25142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T07:13:37.506106+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:13:37.506106+0000","last_peered":"2024-09-13T07:13:37.506106+0000","last_clean":"2024-09-13T07:13:37.506106+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:13:37.506106+0000","last_undegraded":"2024-09-13T07:13:37.506106+0000","last_fullsized":"2024-09-13T07:13:37.506106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"76'425","reported_seq":4936,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.608295+0000","last_change":"2024-09-13T07:13:38.529792+0000","last_active":"2024-09-13T07:13:38.608295+0000","last_peered":"2024-09-13T07:13:38.608295+0000","last_clean":"2024-09-13T07:13:38.608295+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:13:38.608295+0000","last_undegraded":"2024-09-13T07:13:38.608295+0000","last_fullsized":"2024-09-13T07:13:38.608295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.018528111999999999,"stat_sum":{"num_bytes":52526856,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":5658,"num_read_kb":26644,"num_write":623,"num_write_kb":28888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"1.0","version":"0'0","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T07:13:35.555350+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:13:35.555350+0000","last_peered":"2024-09-13T07:13:35.555350+0000","last_clean":"2024-09-13T07:13:35.555350+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:13:35.555350+0000","last_undegraded":"2024-09-13T07:13:35.555350+0000","last_fullsized":"2024-09-13T07:13:35.555350+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'397","reported_seq":2025,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.601426+0000","last_change":"2024-09-13T07:13:38.530422+0000","last_active":"2024-09-13T07:13:38.601426+0000","last_peered":"2024-09-13T07:13:38.601426+0000","last_clean":"2024-09-13T07:13:38.601426+0000","last_became_active":"2024-09-13T07:13:28.445371+0000","last_became_peered":"2024-09-13T07:13:28.445371+0000","last_unstale":"2024-09-13T07:13:38.601426+0000","last_undegraded":"2024-09-13T07:13:38.601426+0000","last_fullsized":"2024-09-13T07:13:38.601426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:22.087974+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.020604900999999998,"stat_sum":{"num_bytes":34029511,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":21,"num_read":2622,"num_read_kb":13131,"num_write":483,"num_write_kb":9265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.9","version":"76'487","reported_seq":1312,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.638914+0000","last_change":"2024-09-13T07:13:38.510436+0000","last_active":"2024-09-13T07:13:38.638914+0000","last_peered":"2024-09-13T07:13:38.638914+0000","last_clean":"2024-09-13T07:13:38.638914+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:13:38.638914+0000","last_undegraded":"2024-09-13T07:13:38.638914+0000","last_fullsized":"2024-09-13T07:13:38.638914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057754900000000001,"stat_sum":{"num_bytes":14495775,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":2257,"num_read_kb":8184,"num_write":743,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.a","version":"76'701","reported_seq":3097,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.615204+0000","last_change":"2024-09-13T07:13:38.540355+0000","last_active":"2024-09-13T07:13:38.615204+0000","last_peered":"2024-09-13T07:13:38.615204+0000","last_clean":"2024-09-13T07:13:38.615204+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:13:38.615204+0000","last_undegraded":"2024-09-13T07:13:38.615204+0000","last_fullsized":"2024-09-13T07:13:38.615204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0061415970000000004,"stat_sum":{"num_bytes":33248985,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":5889,"num_read_kb":19002,"num_write":907,"num_write_kb":9367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.b","version":"76'758","reported_seq":3388,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.601715+0000","last_change":"2024-09-13T07:13:38.535901+0000","last_active":"2024-09-13T07:13:38.601715+0000","last_peered":"2024-09-13T07:13:38.601715+0000","last_clean":"2024-09-13T07:13:38.601715+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:13:38.601715+0000","last_undegraded":"2024-09-13T07:13:38.601715+0000","last_fullsized":"2024-09-13T07:13:38.601715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.025352751999999999,"stat_sum":{"num_bytes":34984383,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":6965,"num_read_kb":19690,"num_write":925,"num_write_kb":10262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.c","version":"76'272","reported_seq":2108,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.617567+0000","last_change":"2024-09-13T07:13:38.522397+0000","last_active":"2024-09-13T07:13:38.617567+0000","last_peered":"2024-09-13T07:13:38.617567+0000","last_clean":"2024-09-13T07:13:38.617567+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:13:38.617567+0000","last_undegraded":"2024-09-13T07:13:38.617567+0000","last_fullsized":"2024-09-13T07:13:38.617567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.015445231,"stat_sum":{"num_bytes":29114993,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":2205,"num_read_kb":20072,"num_write":350,"num_write_kb":12259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.d","version":"74'319","reported_seq":2147,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T07:13:36.576200+0000","last_change":"2024-09-13T07:13:36.576200+0000","last_active":"2024-09-13T07:13:36.576200+0000","last_peered":"2024-09-13T07:13:36.576200+0000","last_clean":"2024-09-13T07:13:36.576200+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:13:36.576200+0000","last_undegraded":"2024-09-13T07:13:36.576200+0000","last_fullsized":"2024-09-13T07:13:36.576200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015069650000000001,"stat_sum":{"num_bytes":15377522,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":25,"num_read":2306,"num_read_kb":8374,"num_write":318,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"3"}]},{"pgid":"3.e","version":"73'582","reported_seq":1650,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T07:13:36.575825+0000","last_change":"2024-09-13T07:13:36.576029+0000","last_active":"2024-09-13T07:13:36.575825+0000","last_peered":"2024-09-13T07:13:36.575825+0000","last_clean":"2024-09-13T07:13:36.575825+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:13:36.575825+0000","last_undegraded":"2024-09-13T07:13:36.575825+0000","last_fullsized":"2024-09-13T07:13:36.575825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001502534,"stat_sum":{"num_bytes":32353698,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":20,"num_read":5742,"num_read_kb":25226,"num_write":741,"num_write_kb":9148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"3"}]},{"pgid":"3.f","version":"72'961","reported_seq":7155,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T07:13:36.575475+0000","last_change":"2024-09-13T07:13:36.575685+0000","last_active":"2024-09-13T07:13:36.575475+0000","last_peered":"2024-09-13T07:13:36.575475+0000","last_clean":"2024-09-13T07:13:36.575475+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:13:36.575475+0000","last_undegraded":"2024-09-13T07:13:36.575475+0000","last_fullsized":"2024-09-13T07:13:36.575475+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157627,"stat_sum":{"num_bytes":34154419,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":9935,"num_read_kb":16929,"num_write":1600,"num_write_kb":24976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"3"}]},{"pgid":"3.10","version":"76'396","reported_seq":2225,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.601597+0000","last_change":"2024-09-13T07:13:38.537220+0000","last_active":"2024-09-13T07:13:38.601597+0000","last_peered":"2024-09-13T07:13:38.601597+0000","last_clean":"2024-09-13T07:13:38.601597+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:13:38.601597+0000","last_undegraded":"2024-09-13T07:13:38.601597+0000","last_fullsized":"2024-09-13T07:13:38.601597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0028581539999999999,"stat_sum":{"num_bytes":45138489,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":26,"num_read":3062,"num_read_kb":21163,"num_write":461,"num_write_kb":27808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.11","version":"72'387","reported_seq":3574,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T07:13:36.574517+0000","last_change":"2024-09-13T07:13:36.574673+0000","last_active":"2024-09-13T07:13:36.574517+0000","last_peered":"2024-09-13T07:13:36.574517+0000","last_clean":"2024-09-13T07:13:36.574517+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:13:36.574517+0000","last_undegraded":"2024-09-13T07:13:36.574517+0000","last_fullsized":"2024-09-13T07:13:36.574517+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022209399999999999,"stat_sum":{"num_bytes":37848636,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":3855,"num_read_kb":31805,"num_write":524,"num_write_kb":12315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"3"}]},{"pgid":"3.12","version":"76'254","reported_seq":1681,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T07:13:38.604398+0000","last_change":"2024-09-13T07:13:38.514073+0000","last_active":"2024-09-13T07:13:38.604398+0000","last_peered":"2024-09-13T07:13:38.604398+0000","last_clean":"2024-09-13T07:13:38.604398+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:13:38.604398+0000","last_undegraded":"2024-09-13T07:13:38.604398+0000","last_fullsized":"2024-09-13T07:13:38.604398+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.004703311,"stat_sum":{"num_bytes":25872483,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":31,"num_read":1303,"num_read_kb":13724,"num_write":232,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"5"}]},{"pgid":"3.13","version":"74'571","reported_seq":3699,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T07:13:36.575720+0000","last_change":"2024-09-13T07:13:36.575720+0000","last_active":"2024-09-13T07:13:36.575720+0000","last_peered":"2024-09-13T07:13:36.575720+0000","last_clean":"2024-09-13T07:13:36.575720+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:13:36.575720+0000","last_undegraded":"2024-09-13T07:13:36.575720+0000","last_fullsized":"2024-09-13T07:13:36.575720+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269362,"stat_sum":{"num_bytes":34448773,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":18,"num_read":5261,"num_read_kb":24422,"num_write":700,"num_write_kb":12616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":936112763,"num_objects":1566,"num_object_clones":356,"num_object_copies":3132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1566,"num_whiteouts":753,"num_read":140352,"num_read_kb":621076,"num_write":18424,"num_write_kb":356187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43266048,"data_stored":361576841,"data_compressed":10129370,"data_compressed_allocated":27439104,"data_compressed_original":347632713,"omap_allocated":7126,"internal_metadata":0},"log_size":14404,"ondisk_log_size":14404,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266694,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44488,"kb_used_data":7588,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94327352,"statfs":{"total":96636764160,"available":96591208448,"internally_reserved":0,"allocated":7770112,"data_stored":60020838,"data_compressed":1628869,"data_compressed_allocated":4730880,"data_compressed_original":57818528,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38664,"kb_used_data":3448,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333176,"statfs":{"total":96636764160,"available":96597172224,"internally_reserved":0,"allocated":3530752,"data_stored":25380052,"data_compressed":754444,"data_compressed_allocated":1970176,"data_compressed_original":24370622,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.498}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.436}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.38}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44992,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326848,"statfs":{"total":96636764160,"available":96590692352,"internally_reserved":0,"allocated":5767168,"data_stored":34010979,"data_compressed":864189,"data_compressed_allocated":2727936,"data_compressed_original":31681701,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.37},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.34200000000000003}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85168,"kb_used_data":9708,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94286672,"statfs":{"total":96636764160,"available":96549552128,"internally_reserved":0,"allocated":9940992,"data_stored":79544256,"data_compressed":1997591,"data_compressed_allocated":5808128,"data_compressed_original":76325433,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.40699999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.436}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.29499999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.371}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":5718016,"data_stored":45059069,"data_compressed":1377310,"data_compressed_allocated":2953216,"data_compressed_original":43062790,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.35599999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.315},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.28299999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41599999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.371}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2844,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2912256,"data_stored":12140355,"data_compressed":218232,"data_compressed_allocated":1060864,"data_compressed_original":10884498,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":4780,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":4894720,"data_stored":29880754,"data_compressed":885841,"data_compressed_allocated":2322432,"data_compressed_original":27987948,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.32200000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.28999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.34999999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.38400000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59068,"kb_used_data":9740,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":9973760,"data_stored":80022864,"data_compressed":2419934,"data_compressed_allocated":5939200,"data_compressed_original":76648073,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.3090000000000002,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":4.54,"5min":4.54,"15min":4.54},"last":0.27100000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.436},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41199999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9076736,"data_stored":79608587,"data_compressed":2419934,"data_compressed_allocated":5939200,"data_compressed_original":76648073,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3944448,"data_stored":28876128,"data_compressed":877321,"data_compressed_allocated":2285568,"data_compressed_original":27414508,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2019328,"data_stored":11726097,"data_compressed":218232,"data_compressed_allocated":1060864,"data_compressed_original":10884498,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4825088,"data_stored":44644811,"data_compressed":1377310,"data_compressed_allocated":2953216,"data_compressed_original":43062790,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9048064,"data_stored":79129998,"data_compressed":1997591,"data_compressed_allocated":5808128,"data_compressed_original":76325433,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4874240,"data_stored":33596721,"data_compressed":864189,"data_compressed_allocated":2727936,"data_compressed_original":31681701,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2637824,"data_stored":24965794,"data_compressed":754444,"data_compressed_allocated":1970176,"data_compressed_original":24370622,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6840320,"data_stored":59028705,"data_compressed":1620349,"data_compressed_allocated":4694016,"data_compressed_original":57245088,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:13:42.026 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:42.027+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:42.069 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:42.070 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:42.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:42.108 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:42.109+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:42.210 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:42.211+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665911585000003, 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:13:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:13:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665886270999998, 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:13:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:13:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665851904999997, 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:13:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665823080999999, 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:13:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665804784000002, 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665779978000003, 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665762104999999, 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:13:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665735763999997, 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:13:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665709042000003, 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:13:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665691457000001, 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:13:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665664819, 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:13:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665647358999998, 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665620400999998, 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665594196999997, 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665576205000001, 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:13:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:13:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665484438, 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:13:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665462138000002, 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665431136999999, 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:13:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665406380999997, 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.665388688999997, 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:42.315 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:42.317 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:42.317 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:42.318+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:42.418 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:42.419+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:42.423 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:42.423 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:42.423 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:42.423 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:42.431 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:42.519 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:42.520+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:42.535 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:42.536 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:42.536 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:42.536 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:42.536 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:42.545 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:42.630 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:42.631+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.476221731999999, 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:13:42.693 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.694 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.695 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.696 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:13:42.697 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.698 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3:f56afaab:::rbd_object_map.117375db170a:head [read 0~112] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 28.345898472999998, 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0257515829999999, 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.699 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.700 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346593+0000", 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346603+0000", 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0037999999999999e-05 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.701 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346626+0000", 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3169000000000001e-05 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346638+0000", 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1712999999999999e-05 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346719+0000", 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0980000000000001e-05 2024-09-13T07:13:42.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346730+0000", 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0657e-05 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348732+0000", 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020022690000000001 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348743+0000", 2024-09-13T07:13:42.703 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0431999999999999e-05 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.371984+0000", 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0232416790000001 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372159+0000", 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017503500000000001 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.704 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372347+0000", 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018757299999999999 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96143 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 2859008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.909336077000001, 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043884619, 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.705 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96143 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783094+0000", 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999361 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783161+0000", 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6962e-05 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783169+0000", 2024-09-13T07:13:42.707 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6620000000000005e-06 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783174+0000", 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4269999999999999e-06 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826848+0000", 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043674360000000002 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.708 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826943+0000", 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4864999999999994e-05 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.827042+0000", 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9096000000000001e-05 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96153 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3350528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:42.709 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907447722000001, 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044029318999999997, 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96153 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785048+0000", 2024-09-13T07:13:42.711 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785054+0000", 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8680000000000004e-06 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785059+0000", 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.712 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828723+0000", 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043664082999999999 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828800+0000", 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6612000000000004e-05 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829075+0000", 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027522899999999999 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.713 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96154 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907430472, 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044169422999999999, 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96154 2024-09-13T07:13:42.714 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:42.715 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785065+0000", 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1789999999999998e-06 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785071+0000", 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785075+0000", 2024-09-13T07:13:42.716 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6310000000000002e-06 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829122+0000", 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044046253 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829175+0000", 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3631000000000002e-05 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829233+0000", 2024-09-13T07:13:42.717 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7348e-05 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96155 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907413952000001, 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044335547000000003, 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.718 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96155 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.719 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785079+0000", 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785082+0000", 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2060000000000001e-06 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785087+0000", 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:13:42.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785092+0000", 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5789999999999997e-06 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829271+0000", 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044178807 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829346+0000", 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5072000000000002e-05 2024-09-13T07:13:42.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829415+0000", 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9474000000000006e-05 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96156 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3497984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907397865, 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044473938999999997, 2024-09-13T07:13:42.722 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96156 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.723 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785098+0000", 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0430000000000002e-06 2024-09-13T07:13:42.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785103+0000", 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.446e-06 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785108+0000", 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.725 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829445+0000", 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044337623999999999 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829496+0000", 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0705000000000003e-05 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829570+0000", 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3830000000000003e-05 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96157 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3547136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907382158000001, 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044623329000000003, 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96157 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.727 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785111+0000", 2024-09-13T07:13:42.728 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785113+0000", 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785119+0000", 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6080000000000003e-06 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785124+0000", 2024-09-13T07:13:42.729 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6600000000000003e-06 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829591+0000", 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044467818999999999 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829668+0000", 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6925e-05 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.730 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829735+0000", 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6631000000000006e-05 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96158 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907350597000001, 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044723374000000003, 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.731 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96158 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.732 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785145+0000", 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785152+0000", 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4049999999999997e-06 2024-09-13T07:13:42.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785157+0000", 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.814e-06 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829772+0000", 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044615624 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829823+0000", 2024-09-13T07:13:42.734 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1131000000000002e-05 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829866+0000", 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3237e-05 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96159 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907332837, 2024-09-13T07:13:42.735 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044827964999999997, 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96159 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.736 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785160+0000", 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785163+0000", 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2259999999999999e-06 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.737 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785168+0000", 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6219999999999997e-06 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785173+0000", 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4989999999999996e-06 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829905+0000", 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044732582 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.738 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829949+0000", 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3593000000000001e-05 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829989+0000", 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9919000000000002e-05 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.739 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96160 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3694592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907317296, 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045000912999999997, 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96160 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.740 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.741 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785182+0000", 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4550000000000003e-06 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785188+0000", 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4749999999999998e-06 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785193+0000", 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3109999999999996e-06 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.742 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830026+0000", 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044832982 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830097+0000", 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1074000000000004e-05 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830177+0000", 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9949000000000003e-05 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.743 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96161 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907296299, 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045119541999999999, 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96161 2024-09-13T07:13:42.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.745 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785199+0000", 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1399999999999998e-06 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785205+0000", 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6370000000000004e-06 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.746 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785209+0000", 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830212+0000", 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045002535000000003 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830277+0000", 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5617999999999994e-05 2024-09-13T07:13:42.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830317+0000", 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9617999999999999e-05 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96162 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3792896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907280059, 2024-09-13T07:13:42.748 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196102000000002, 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96162 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.749 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785213+0000", 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785215+0000", 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.069e-06 2024-09-13T07:13:42.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785221+0000", 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6980000000000003e-06 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785226+0000", 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5170000000000001e-06 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830332+0000", 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045106780999999999 2024-09-13T07:13:42.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830375+0000", 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2474e-05 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830410+0000", 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4909000000000001e-05 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.752 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96163 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.907258819000001, 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045359167999999998, 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96163 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.753 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785234+0000", 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.754 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785237+0000", 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7149999999999998e-06 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785243+0000", 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2349999999999996e-06 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785248+0000", 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.78e-06 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.755 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830445+0000", 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196419000000002 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830514+0000", 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9029999999999995e-05 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830594+0000", 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0520999999999997e-05 2024-09-13T07:13:42.756 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96165 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.905584943000001, 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044000064999999998, 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.757 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96165 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.758 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786908+0000", 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786911+0000", 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786916+0000", 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.84e-06 2024-09-13T07:13:42.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786921+0000", 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830763+0000", 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043841106999999997 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.760 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830834+0000", 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1304999999999996e-05 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830909+0000", 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4987999999999994e-05 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96166 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:13:42.761 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.905567873000001, 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044109809, 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96166 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786925+0000", 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786928+0000", 2024-09-13T07:13:42.763 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0820000000000001e-06 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786933+0000", 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786938+0000", 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.764 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830944+0000", 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044005374999999999 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830993+0000", 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8900000000000003e-05 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831036+0000", 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3139000000000002e-05 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.765 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96167 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.905551552, 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044213595000000001, 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.766 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96167 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.767 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786944+0000", 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3829999999999999e-06 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786950+0000", 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:13:42.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786955+0000", 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6059999999999996e-06 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831062+0000", 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044107252999999999 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.769 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831101+0000", 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9175999999999999e-05 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831156+0000", 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4586999999999998e-05 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.770 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96168 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.905535417999999, 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044324181999999997, 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96168 2024-09-13T07:13:42.771 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.772 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786960+0000", 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0789999999999999e-06 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786966+0000", 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6330000000000001e-06 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.773 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786970+0000", 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831185+0000", 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044215245 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831231+0000", 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5952000000000003e-05 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.774 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831282+0000", 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1258999999999998e-05 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96169 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4136960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.905520028, 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044443489000000003, 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.775 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96169 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.776 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786973+0000", 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786975+0000", 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.777 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786981+0000", 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4389999999999999e-06 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786985+0000", 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5929999999999999e-06 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831321+0000", 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044336028 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.778 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831380+0000", 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8329999999999999e-05 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831417+0000", 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7361999999999999e-05 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96170 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:42.779 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 14.905495564000001, 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523786000000003, 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96170 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.780 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787001+0000", 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.932e-06 2024-09-13T07:13:42.781 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787007+0000", 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1739999999999997e-06 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787012+0000", 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.899e-06 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831437+0000", 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044425112000000003 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.782 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831480+0000", 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2864000000000002e-05 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831522+0000", 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2366000000000001e-05 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:42.783 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:42.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:42.785 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:42.786+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:42.885 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:42.886+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:42.928 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:42.928 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:42.928 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:42.928 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:42.937 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:42.986 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:42.987+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:43.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:43.038 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:43.038 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:43.038 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:43.038 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:43.038 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:43.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:43.087 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:43.088+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:43.145 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:43.168 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:43.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:43.188 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:43.189+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:43.201 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:43.202 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.539246032999998, 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:43.203 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:43.204 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:43.205 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:43.206 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.539208381999998, 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:43.207 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:43.208 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:43.209 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.210 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.539189155000003, 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:43.211 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:43.212 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.213 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.214 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.663115742999999, 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.215 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.216 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.217 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.218 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:43.219 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.220 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96967 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354178421, 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028701085000000001, 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96967 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.221 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848739+0000", 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848741+0000", 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.153e-06 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848743+0000", 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.723e-06 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848746+0000", 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.858e-06 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877369+0000", 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028623282999999999 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877406+0000", 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6139000000000001e-05 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877441+0000", 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151000000000003e-05 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.224 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96968 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354169860000001, 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028858604999999999, 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.225 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96968 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848749+0000", 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.091e-06 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848752+0000", 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7769999999999998e-06 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848754+0000", 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2869999999999999e-06 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877468+0000", 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028714185 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877541+0000", 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2927000000000002e-05 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877607+0000", 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5560000000000002e-05 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96969 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354161712, 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028944845, 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.229 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96969 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848757+0000", 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848761+0000", 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848763+0000", 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.311e-06 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877630+0000", 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866326000000001 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877666+0000", 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6127000000000002e-05 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877701+0000", 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5413000000000001e-05 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96970 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354152721, 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029100663999999998, 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96970 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848766+0000", 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.097e-06 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848769+0000", 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848772+0000", 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4269999999999998e-06 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877729+0000", 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028957488 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877800+0000", 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1113999999999997e-05 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877866+0000", 2024-09-13T07:13:43.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5882000000000004e-05 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96971 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354144591000001, 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029181689, 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.237 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96971 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:43.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848774+0000", 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1209999999999999e-06 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848777+0000", 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.926e-06 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848780+0000", 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:13:43.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877884+0000", 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029104076999999999 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877919+0000", 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5655999999999999e-05 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877955+0000", 2024-09-13T07:13:43.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5782999999999999e-05 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96972 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.3541364, 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029338843999999999, 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.241 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96972 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848781+0000", 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848783+0000", 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1659999999999999e-06 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848785+0000", 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.818e-06 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848788+0000", 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4569999999999999e-06 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877984+0000", 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029196203 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878055+0000", 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0622000000000002e-05 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878120+0000", 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5816999999999997e-05 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96973 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354111230000001, 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029407188000000001, 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.245 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96973 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848806+0000", 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848808+0000", 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7039999999999999e-06 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848813+0000", 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6360000000000002e-06 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848815+0000", 2024-09-13T07:13:43.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5799999999999999e-06 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878143+0000", 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029327678999999999 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878179+0000", 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5587000000000003e-05 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878214+0000", 2024-09-13T07:13:43.248 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5456000000000001e-05 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96974 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354099931, 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029557028999999999, 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.249 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96974 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:43.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848821+0000", 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5140000000000002e-06 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848825+0000", 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848827+0000", 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.39e-06 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878240+0000", 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029413174 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878310+0000", 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9813000000000004e-05 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878375+0000", 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5122000000000004e-05 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96975 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.35408889, 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029640264, 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.253 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96975 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848830+0000", 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.235e-06 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848833+0000", 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.796e-06 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.255 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848835+0000", 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.402e-06 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878397+0000", 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029561548999999999 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878434+0000", 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6721000000000003e-05 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.256 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878469+0000", 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5855999999999997e-05 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96976 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354080769999999, 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029795095000000001, 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96976 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848838+0000", 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.156e-06 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848841+0000", 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9110000000000002e-06 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848843+0000", 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878496+0000", 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029652371 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878567+0000", 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0784000000000005e-05 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878632+0000", 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5783000000000004e-05 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96977 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354072639, 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029869717, 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.261 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96977 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848846+0000", 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1000000000000001e-06 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848849+0000", 2024-09-13T07:13:43.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.835e-06 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848851+0000", 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878643+0000", 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029791405 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878678+0000", 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5139999999999999e-05 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878715+0000", 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7119000000000002e-05 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96978 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:43.265 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354064621999999, 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030029390999999999, 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96978 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:43.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848854+0000", 2024-09-13T07:13:43.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1039999999999999e-06 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848857+0000", 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8399999999999999e-06 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848859+0000", 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878742+0000", 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029882140000000001 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878813+0000", 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1771000000000001e-05 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878883+0000", 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9457999999999998e-05 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96979 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.35405669, 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030106193999999999, 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96979 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848862+0000", 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.234e-06 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848865+0000", 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.836e-06 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.272 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848868+0000", 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878895+0000", 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030027642 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878931+0000", 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5624999999999999e-05 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878968+0000", 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6720000000000001e-05 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96980 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354048531, 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030267181000000001, 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.274 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96980 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848869+0000", 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.088e-06 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848873+0000", 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9019999999999999e-06 2024-09-13T07:13:43.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848875+0000", 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1950000000000002e-06 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878996+0000", 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03012068 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879067+0000", 2024-09-13T07:13:43.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1100999999999996e-05 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879137+0000", 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9424999999999993e-05 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96981 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354034711000001, 2024-09-13T07:13:43.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030344368, 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96981 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848885+0000", 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848888+0000", 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848891+0000", 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.554e-06 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879159+0000", 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030268155000000001 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879192+0000", 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879228+0000", 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6146999999999998e-05 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96982 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:43.282 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 15.354025446, 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030488562, 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96982 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:43.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848892+0000", 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:43.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848894+0000", 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2079999999999999e-06 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848896+0000", 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8679999999999999e-06 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848899+0000", 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:43.285 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879246+0000", 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030346838000000001 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879317+0000", 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1254000000000006e-05 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879381+0000", 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4331000000000005e-05 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:43.286 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:43.287 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:43.287 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:43.288 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:43.288 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:43.288 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:43.289 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:43.289 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:43.290 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:43.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:43.291 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:43.329 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:43.330+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:43.393 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:43.394+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:43.430 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:43.430 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:43.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:43.493 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:43.494+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:43.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:13:43.581 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:43.582 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:43.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:43.634 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:43.635+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:43.693 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:43.693 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:43.697 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:43.698+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:43.702 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:43.798 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:43.799+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:43.845 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:43.845 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:43.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:43.939 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:43.940+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:43.997 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:43.999 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:44.000+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:44.006 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:44.100 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:44.101+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:44.145 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:13:44.158 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:44.158 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:44.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:44.212 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:44.213+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:44.301 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:44.303+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:44.327 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:44.328 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:44.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:44.402 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:44.403+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:44.443 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:44.443 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:44.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:44.536 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:44.537+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:44.599 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:44.600 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:44.604 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:44.605+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:44.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:44.705 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:44.706+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:44.755 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:44.756 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:44.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:44.808 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:44.809+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:44.870 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:44.870 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:44.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:44.921 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:44.922+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:45.007 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:45.008+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:45.027 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:45.027 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:45.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:45.109 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:45.110+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:45.188 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:45.188 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:45.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:45.209 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:45.210+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:45.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:45.349 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:45.349 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:45.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:45.350 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:45.349+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:45.449 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:45.450+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:45.497 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:45.498 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:45.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:45.513 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:45.514 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:45.530 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:45.530 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:45.548 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:45.550 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:45.551+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:45.564 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:45.581 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:45.598 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:45.615 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:45.615 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:45.621 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:45.621 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:45.621 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:45.622 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:45.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:45.632 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:45.632 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:45.685 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:45.686+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:45.739 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:45.739 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:45.739 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:45.739 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:45.739 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:45.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:45.751 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:45.752+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:45.852 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:45.854+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:45.895 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:45.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:45.895 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:45.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.334120867000003, 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:13:45.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.334095552999997, 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.334061187000003, 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:13:45.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.334032362999999, 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:13:45.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.334014066000002, 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:13:45.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333989260000003, 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:13:45.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:13:45.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333971386999998, 2024-09-13T07:13:45.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:13:45.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333945045999997, 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:13:45.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333918324000003, 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:13:45.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:13:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333900739000001, 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:13:45.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333874100999999, 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:13:45.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:13:45.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333856640999997, 2024-09-13T07:13:45.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:45.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333829682999998, 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:13:45.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333803478999997, 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:13:45.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:13:45.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333785487, 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:13:45.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:13:45.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333693719999999, 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:13:45.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:45.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333671420000002, 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:13:45.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333640418999998, 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:13:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333615663000003, 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:13:45.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.333597971000003, 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:45.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:13:45.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:45.984 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:45.986 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:45.986 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:45.987+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:46.086 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:46.087+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:46.103 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:46.103 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:46.103 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:46.103 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:46.112 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:46.173 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:13:46.187 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:46.188+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:46.226 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:46.226 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:46.226 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:46.226 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:46.226 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:46.236 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:46.288 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:46.288+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:46.381 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:46.381 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:46.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.164920271, 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:13:46.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:46.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:13:46.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:13:46.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.387 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3:f56afaab:::rbd_object_map.117375db170a:head [read 0~112] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 32.034597011999999, 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0257515829999999, 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346593+0000", 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346603+0000", 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0037999999999999e-05 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346626+0000", 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3169000000000001e-05 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346638+0000", 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1712999999999999e-05 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346719+0000", 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0980000000000001e-05 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346730+0000", 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0657e-05 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348732+0000", 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020022690000000001 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348743+0000", 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0431999999999999e-05 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.371984+0000", 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0232416790000001 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372159+0000", 2024-09-13T07:13:46.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017503500000000001 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372347+0000", 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018757299999999999 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96143 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 2859008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:13:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.598034616, 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043884619, 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96143 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783094+0000", 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999361 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783161+0000", 2024-09-13T07:13:46.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6962e-05 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783169+0000", 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6620000000000005e-06 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783174+0000", 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4269999999999999e-06 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826848+0000", 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043674360000000002 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826943+0000", 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4864999999999994e-05 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.827042+0000", 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9096000000000001e-05 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.397 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96153 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3350528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596146261000001, 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044029318999999997, 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96153 2024-09-13T07:13:46.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785048+0000", 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785054+0000", 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8680000000000004e-06 2024-09-13T07:13:46.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785059+0000", 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828723+0000", 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043664082999999999 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828800+0000", 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6612000000000004e-05 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829075+0000", 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027522899999999999 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96154 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:46.402 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596129010999999, 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044169422999999999, 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96154 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785065+0000", 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1789999999999998e-06 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785071+0000", 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785075+0000", 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6310000000000002e-06 2024-09-13T07:13:46.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829122+0000", 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044046253 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829175+0000", 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3631000000000002e-05 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829233+0000", 2024-09-13T07:13:46.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7348e-05 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96155 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596112491, 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044335547000000003, 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.407 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96155 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785079+0000", 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785082+0000", 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2060000000000001e-06 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785087+0000", 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785092+0000", 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5789999999999997e-06 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829271+0000", 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044178807 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829346+0000", 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5072000000000002e-05 2024-09-13T07:13:46.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829415+0000", 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9474000000000006e-05 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96156 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3497984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596096404000001, 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044473938999999997, 2024-09-13T07:13:46.411 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96156 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785098+0000", 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0430000000000002e-06 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785103+0000", 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.446e-06 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785108+0000", 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829445+0000", 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044337623999999999 2024-09-13T07:13:46.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829496+0000", 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0705000000000003e-05 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829570+0000", 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3830000000000003e-05 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.415 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96157 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3547136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596080697000001, 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044623329000000003, 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96157 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785111+0000", 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785113+0000", 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785119+0000", 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6080000000000003e-06 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785124+0000", 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6600000000000003e-06 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829591+0000", 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044467818999999999 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829668+0000", 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6925e-05 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829735+0000", 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6631000000000006e-05 2024-09-13T07:13:46.419 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96158 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596049136000001, 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044723374000000003, 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.420 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96158 2024-09-13T07:13:46.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.421 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:46.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785145+0000", 2024-09-13T07:13:46.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785152+0000", 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4049999999999997e-06 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785157+0000", 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.814e-06 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829772+0000", 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044615624 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829823+0000", 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1131000000000002e-05 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829866+0000", 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3237e-05 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.424 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96159 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596031375999999, 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044827964999999997, 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96159 2024-09-13T07:13:46.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785160+0000", 2024-09-13T07:13:46.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785163+0000", 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2259999999999999e-06 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785168+0000", 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6219999999999997e-06 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785173+0000", 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4989999999999996e-06 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829905+0000", 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044732582 2024-09-13T07:13:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829949+0000", 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3593000000000001e-05 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829989+0000", 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9919000000000002e-05 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96160 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3694592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.596015834999999, 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045000912999999997, 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96160 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.430 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785182+0000", 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4550000000000003e-06 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785188+0000", 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4749999999999998e-06 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785193+0000", 2024-09-13T07:13:46.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3109999999999996e-06 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830026+0000", 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044832982 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830097+0000", 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1074000000000004e-05 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830177+0000", 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9949000000000003e-05 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96161 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.595994837999999, 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045119541999999999, 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96161 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785199+0000", 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1399999999999998e-06 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785205+0000", 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6370000000000004e-06 2024-09-13T07:13:46.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785209+0000", 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830212+0000", 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045002535000000003 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830277+0000", 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5617999999999994e-05 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830317+0000", 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9617999999999999e-05 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96162 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3792896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.595978597999999, 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196102000000002, 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96162 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:13:46.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785213+0000", 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785215+0000", 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.069e-06 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785221+0000", 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6980000000000003e-06 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785226+0000", 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5170000000000001e-06 2024-09-13T07:13:46.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830332+0000", 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045106780999999999 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830375+0000", 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2474e-05 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830410+0000", 2024-09-13T07:13:46.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4909000000000001e-05 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96163 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.595957358, 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045359167999999998, 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96163 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785234+0000", 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785237+0000", 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7149999999999998e-06 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785243+0000", 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2349999999999996e-06 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785248+0000", 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.78e-06 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830445+0000", 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196419000000002 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830514+0000", 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9029999999999995e-05 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830594+0000", 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0520999999999997e-05 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96165 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:13:46.447 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.594283482000002, 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044000064999999998, 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96165 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786908+0000", 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786911+0000", 2024-09-13T07:13:46.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786916+0000", 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.84e-06 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786921+0000", 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830763+0000", 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043841106999999997 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830834+0000", 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1304999999999996e-05 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830909+0000", 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4987999999999994e-05 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.451 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96166 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.594266412, 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044109809, 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96166 2024-09-13T07:13:46.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786925+0000", 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786928+0000", 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0820000000000001e-06 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786933+0000", 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786938+0000", 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830944+0000", 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044005374999999999 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830993+0000", 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8900000000000003e-05 2024-09-13T07:13:46.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831036+0000", 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3139000000000002e-05 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96167 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:46.456 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.594250090999999, 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044213595000000001, 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96167 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:46.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786944+0000", 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3829999999999999e-06 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786950+0000", 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786955+0000", 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6059999999999996e-06 2024-09-13T07:13:46.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831062+0000", 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044107252999999999 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831101+0000", 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9175999999999999e-05 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831156+0000", 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4586999999999998e-05 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96168 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.594233957, 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044324181999999997, 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96168 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786960+0000", 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0789999999999999e-06 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786966+0000", 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6330000000000001e-06 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786970+0000", 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831185+0000", 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044215245 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831231+0000", 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5952000000000003e-05 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831282+0000", 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1258999999999998e-05 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96169 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4136960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.594218566999999, 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044443489000000003, 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.465 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96169 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786973+0000", 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786975+0000", 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786981+0000", 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4389999999999999e-06 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786985+0000", 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5929999999999999e-06 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831321+0000", 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044336028 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831380+0000", 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8329999999999999e-05 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831417+0000", 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7361999999999999e-05 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.469 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96170 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 18.594194103, 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523786000000003, 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96170 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:13:46.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787001+0000", 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.932e-06 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787007+0000", 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1739999999999997e-06 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787012+0000", 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.899e-06 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831437+0000", 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044425112000000003 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831480+0000", 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2864000000000002e-05 2024-09-13T07:13:46.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831522+0000", 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2366000000000001e-05 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.474 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:46.476 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:46.476 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:46.477+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:46.577 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:46.578+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:46.618 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:46.618 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:46.618 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:46.618 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:46.626 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:46.678 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:46.679+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:46.770 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:46.770 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:46.770 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:46.770 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:46.770 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:46.778 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:46.779 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:46.780+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:46.879 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:46.880+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:46.882 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:46.882 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:46.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:46.882 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:46.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.882 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.218977694000003, 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:46.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.218940043000003, 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.887 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:46.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:46.891 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.218920816000001, 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.342847403999997, 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:46.895 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:46.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:46.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:46.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96967 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:46.901 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033910081999998, 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028701085000000001, 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96967 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:46.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848739+0000", 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848741+0000", 2024-09-13T07:13:46.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.153e-06 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848743+0000", 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.723e-06 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848746+0000", 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.858e-06 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877369+0000", 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028623282999999999 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877406+0000", 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6139000000000001e-05 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877441+0000", 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151000000000003e-05 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.905 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96968 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033901521000001, 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028858604999999999, 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96968 2024-09-13T07:13:46.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848749+0000", 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.091e-06 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848752+0000", 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7769999999999998e-06 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848754+0000", 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2869999999999999e-06 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877468+0000", 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028714185 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877541+0000", 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2927000000000002e-05 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877607+0000", 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5560000000000002e-05 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96969 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033893373000002, 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028944845, 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.910 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96969 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848757+0000", 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848761+0000", 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848763+0000", 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.311e-06 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877630+0000", 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866326000000001 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877666+0000", 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6127000000000002e-05 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877701+0000", 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5413000000000001e-05 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96970 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.914 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033884382, 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029100663999999998, 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96970 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848766+0000", 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.097e-06 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848769+0000", 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848772+0000", 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4269999999999998e-06 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877729+0000", 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028957488 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877800+0000", 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1113999999999997e-05 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877866+0000", 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5882000000000004e-05 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96971 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033876251999999, 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029181689, 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.919 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96971 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848774+0000", 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1209999999999999e-06 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848777+0000", 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.926e-06 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848780+0000", 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877884+0000", 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029104076999999999 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877919+0000", 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5655999999999999e-05 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877955+0000", 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5782999999999999e-05 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.923 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96972 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033868061, 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029338843999999999, 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96972 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848781+0000", 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848783+0000", 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1659999999999999e-06 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848785+0000", 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.818e-06 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848788+0000", 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4569999999999999e-06 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877984+0000", 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029196203 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878055+0000", 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0622000000000002e-05 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878120+0000", 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5816999999999997e-05 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.927 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96973 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033842890999999, 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029407188000000001, 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96973 2024-09-13T07:13:46.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848806+0000", 2024-09-13T07:13:46.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848808+0000", 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7039999999999999e-06 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848813+0000", 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6360000000000002e-06 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848815+0000", 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5799999999999999e-06 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878143+0000", 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029327678999999999 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878179+0000", 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5587000000000003e-05 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878214+0000", 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5456000000000001e-05 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96974 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033831591999999, 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029557028999999999, 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96974 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848821+0000", 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5140000000000002e-06 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848825+0000", 2024-09-13T07:13:46.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848827+0000", 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.39e-06 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878240+0000", 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029413174 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878310+0000", 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9813000000000004e-05 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878375+0000", 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5122000000000004e-05 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96975 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.936 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033820551000002, 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029640264, 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96975 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848830+0000", 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.235e-06 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848833+0000", 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.796e-06 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848835+0000", 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.402e-06 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878397+0000", 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029561548999999999 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878434+0000", 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6721000000000003e-05 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878469+0000", 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5855999999999997e-05 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96976 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033812431000001, 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029795095000000001, 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.941 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96976 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848838+0000", 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.156e-06 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848841+0000", 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9110000000000002e-06 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848843+0000", 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878496+0000", 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029652371 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878567+0000", 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0784000000000005e-05 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878632+0000", 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5783000000000004e-05 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96977 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.945 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.0338043, 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029869717, 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96977 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848846+0000", 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1000000000000001e-06 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848849+0000", 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.835e-06 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848851+0000", 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:46.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878643+0000", 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029791405 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878678+0000", 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5139999999999999e-05 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878715+0000", 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7119000000000002e-05 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96978 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033796283000001, 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030029390999999999, 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.950 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96978 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848854+0000", 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1039999999999999e-06 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.952 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848857+0000", 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8399999999999999e-06 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848859+0000", 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878742+0000", 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029882140000000001 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878813+0000", 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1771000000000001e-05 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878883+0000", 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9457999999999998e-05 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.954 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96979 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033788350999998, 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030106193999999999, 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96979 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:46.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848862+0000", 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.234e-06 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848865+0000", 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.836e-06 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848868+0000", 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878895+0000", 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030027642 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878931+0000", 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5624999999999999e-05 2024-09-13T07:13:46.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878968+0000", 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6720000000000001e-05 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96980 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033780191999998, 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030267181000000001, 2024-09-13T07:13:46.959 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96980 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848869+0000", 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.088e-06 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848873+0000", 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9019999999999999e-06 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848875+0000", 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1950000000000002e-06 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878996+0000", 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03012068 2024-09-13T07:13:46.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879067+0000", 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1100999999999996e-05 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879137+0000", 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9424999999999993e-05 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96981 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.963 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033766371999999, 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030344368, 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96981 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848885+0000", 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:46.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848888+0000", 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848891+0000", 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.554e-06 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879159+0000", 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030268155000000001 2024-09-13T07:13:46.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879192+0000", 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879228+0000", 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6146999999999998e-05 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96982 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.033757107, 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030488562, 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96982 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.968 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848892+0000", 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:46.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848894+0000", 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2079999999999999e-06 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848896+0000", 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8679999999999999e-06 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848899+0000", 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-13T07:13:46.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879246+0000", 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030346838000000001 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879317+0000", 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1254000000000006e-05 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879381+0000", 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4331000000000005e-05 2024-09-13T07:13:46.971 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.972 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.972 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.972 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:46.972 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:46.972 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:46.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:46.988 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:46.988 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:46.988 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T07:13:46.988 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:13:46.988 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:13:46.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T07:13:47.014 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:47.015+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:47.081 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:47.081+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:47.117 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:47.117 INFO:teuthology.orchestra.run.smithi184.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:13:47.117 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:47.118 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:47.118 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:47.118 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:47.118 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:47.128 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:13:47.217 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:47.218+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:47.276 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:47.276 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:47.277 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:47.277 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:47.277 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:47.282 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:47.283+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:47.285 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:47.383 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:47.384+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 69.391820202999995, 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:13:47.429 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.844911330999999, 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:47.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88519 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.843753497000002, 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017910447999999999, 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.440 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88519 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585257+0000", 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585267+0000", 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0635000000000001e-05 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585289+0000", 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1846999999999999e-05 2024-09-13T07:13:47.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585299+0000", 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4949999999999997e-06 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585347+0000", 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7672e-05 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585500+0000", 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015363700000000001 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.588596+0000", 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030953669999999999 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603068+0000", 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014472063 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:47.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603106+0000", 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8435e-05 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603142+0000", 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6488e-05 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603169+0000", 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6959000000000001e-05 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:89021 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.899276583999999, 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02102269, 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 89021 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529734+0000", 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529742+0000", 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8290000000000008e-06 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529757+0000", 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4868999999999999e-05 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529763+0000", 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.93e-06 2024-09-13T07:13:47.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529803+0000", 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0404999999999997e-05 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529809+0000", 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531015+0000", 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012062049999999999 2024-09-13T07:13:47.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531057+0000", 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1921000000000001e-05 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.534526+0000", 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034694909999999999 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550683+0000", 2024-09-13T07:13:47.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016157169999999998 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550697+0000", 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3351e-05 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550731+0000", 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3997e-05 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550759+0000", 2024-09-13T07:13:47.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7861e-05 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95026 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.733262518, 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014972104999999999, 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95026 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695752+0000", 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.283e-06 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695758+0000", 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:13:47.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695762+0000", 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710581+0000", 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014818934000000001 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710653+0000", 2024-09-13T07:13:47.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2457000000000001e-05 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710722+0000", 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8633999999999994e-05 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95027 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.733246408999999, 2024-09-13T07:13:47.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015079646, 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95027 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695768+0000", 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:13:47.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695773+0000", 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.592e-06 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695778+0000", 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710749+0000", 2024-09-13T07:13:47.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014970809 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710805+0000", 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6149000000000001e-05 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710846+0000", 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0840999999999997e-05 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.460 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95028 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.733230713000001, 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015231216000000001, 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95028 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695781+0000", 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695783+0000", 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9250000000000002e-06 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695789+0000", 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.835e-06 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695793+0000", 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3000000000000003e-06 2024-09-13T07:13:47.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710881+0000", 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015087695999999999 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710947+0000", 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6169999999999998e-05 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.711013+0000", 2024-09-13T07:13:47.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5643999999999997e-05 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95080 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731629586, 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014948011000000001, 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.465 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95080 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697385+0000", 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.187e-06 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697388+0000", 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9459999999999998e-06 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697390+0000", 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5600000000000001e-06 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712211+0000", 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014820224 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712276+0000", 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5650999999999999e-05 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712331+0000", 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4727e-05 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95081 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.469 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.73162013, 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015124461000000001, 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95081 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:47.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697394+0000", 2024-09-13T07:13:47.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.418e-06 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697397+0000", 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697399+0000", 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.508e-06 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712367+0000", 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01496809 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712442+0000", 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5290999999999995e-05 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712517+0000", 2024-09-13T07:13:47.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4396999999999999e-05 2024-09-13T07:13:47.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.476 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95082 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731611445999999, 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015249593000000001, 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95082 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:47.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697402+0000", 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.311e-06 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697405+0000", 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697407+0000", 2024-09-13T07:13:47.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712557+0000", 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015149207 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712609+0000", 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2286e-05 2024-09-13T07:13:47.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712651+0000", 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1687999999999998e-05 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95083 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3727360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731602956, 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015418032, 2024-09-13T07:13:47.481 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95083 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:47.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697409+0000", 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.049e-06 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697414+0000", 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.783e-06 2024-09-13T07:13:47.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697417+0000", 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.587e-06 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712683+0000", 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015266083 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712759+0000", 2024-09-13T07:13:47.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6627999999999999e-05 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712828+0000", 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8121999999999996e-05 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95084 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3776512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731593685, 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015560297000000001, 2024-09-13T07:13:47.485 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95084 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697420+0000", 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697423+0000", 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3359999999999999e-06 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697425+0000", 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.452e-06 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712864+0000", 2024-09-13T07:13:47.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015438927999999999 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712938+0000", 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3458e-05 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712979+0000", 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1251999999999997e-05 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95085 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731582894999999, 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015726161999999998, 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95085 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697429+0000", 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697431+0000", 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5230000000000001e-06 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697434+0000", 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697436+0000", 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3889999999999999e-06 2024-09-13T07:13:47.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713008+0000", 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015571295000000001 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713084+0000", 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6537000000000003e-05 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713156+0000", 2024-09-13T07:13:47.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1616e-05 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95086 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3874816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.73157389, 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015894697999999999, 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95086 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697438+0000", 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697440+0000", 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.159e-06 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697443+0000", 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9359999999999999e-06 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697445+0000", 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713191+0000", 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015746343999999999 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713269+0000", 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7737000000000005e-05 2024-09-13T07:13:47.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713333+0000", 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4233000000000007e-05 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95087 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731565550999999, 2024-09-13T07:13:47.498 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01604939, 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95087 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.499 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697448+0000", 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.094e-06 2024-09-13T07:13:47.500 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697451+0000", 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8949999999999998e-06 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697454+0000", 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.999e-06 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.501 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713361+0000", 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015907092000000001 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713433+0000", 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2606000000000002e-05 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713496+0000", 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2908000000000001e-05 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95088 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731555410999999, 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016219289000000001, 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95088 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:47.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697459+0000", 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.764e-06 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697462+0000", 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4240000000000002e-06 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697464+0000", 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3989999999999998e-06 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713532+0000", 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016067403000000001 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713607+0000", 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5476999999999996e-05 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713676+0000", 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9115999999999994e-05 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95089 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731545558000001, 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016381144, 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95089 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.507 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:47.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697468+0000", 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.734e-06 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697474+0000", 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697479+0000", 2024-09-13T07:13:47.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713703+0000", 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016224643 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4828999999999998e-05 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713848+0000", 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0029000000000003e-05 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95090 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731529760000001, 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016573104000000002, 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.511 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95090 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697482+0000", 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697484+0000", 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.916e-06 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697496+0000", 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697506+0000", 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0477e-05 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713903+0000", 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016396758000000001 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713981+0000", 2024-09-13T07:13:47.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7892999999999994e-05 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714056+0000", 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5214000000000002e-05 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95091 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731496339, 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016674768, 2024-09-13T07:13:47.515 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95091 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697518+0000", 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4789999999999999e-06 2024-09-13T07:13:47.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697527+0000", 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3799999999999994e-06 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697532+0000", 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714080+0000", 2024-09-13T07:13:47.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016547798999999998 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714135+0000", 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5294000000000003e-05 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714191+0000", 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5606999999999998e-05 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95092 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4169728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 19.731475550999999, 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017373829, 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95092 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.520 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697536+0000", 2024-09-13T07:13:47.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697540+0000", 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4879999999999999e-06 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697548+0000", 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0139999999999998e-06 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:47.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697553+0000", 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2460000000000003e-06 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714754+0000", 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017200653 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714831+0000", 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7156000000000005e-05 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:47.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714911+0000", 2024-09-13T07:13:47.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9740999999999994e-05 2024-09-13T07:13:47.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.524 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.524 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:47.524 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:47.524 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:47.525 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:47.526 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:47.527+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:47.626 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:47.626+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:47.644 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:47.668 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:47.668 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:47.668 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:47.668 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:47.677 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:47.726 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:47.727+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:47.776 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:47.780 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:47.780 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:47.780 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:47.780 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:47.780 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:47.789 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:47.792 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:47.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:47.808 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:47.824 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:47.827 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:47.828+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:47.842 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:47.858 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:47.859 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:47.875 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:47.875 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:47.893 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:47.893 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:47.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:47.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:47.928 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:47.929+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:48.008 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.008 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:48.061 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:48.062+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:48.123 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.124 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.129 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:48.130+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:48.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:48.230 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:48.231+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:48.290 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.290 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:48.342 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:48.343+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:48.405 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.405 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:48.431 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:48.433+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:48.533 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:48.534+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:48.561 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.561 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:48.634 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:48.634+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:48.676 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.676 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:48.734 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:48.735+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:48.793 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.793 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:48.843 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:48.844+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:48.936 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:48.937+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:48.949 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:48.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:49.043 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:49.044+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:49.104 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:49.104 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:49.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:49.155 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:49.156+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:49.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:13:49.238 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:49.239+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:49.276 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:49.276 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:49.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:49.339 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:49.340+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:49.395 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:49.396 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:49.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:49.440 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:49.441+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:49.541 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:49.542+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:49.550 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:49.551 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:49.559 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:49.642 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:49.643+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:49.663 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:49.663 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:49.672 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:49.756 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:49.757+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:49.815 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:49.815 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:49.824 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:49.865 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:49.866+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:49.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:49.950 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:49.951+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:49.966 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:49.966 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:49.975 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:50.033 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:50.033 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:50.045 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:50.046+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:50.050 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:50.050 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:50.065 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:50.078 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:50.078 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:50.081 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:50.087 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:50.098 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:50.115 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:50.131 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:50.146 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:50.147+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:50.148 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:50.148 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:13:50.242 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:50.242 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:50.242 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:50.247 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:50.248+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:50.252 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:50.349 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:50.350+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:50.398 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:50.398 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:50.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:50.398 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:50.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.398 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:50.398 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 48.733861713000003, 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:13:50.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:13:50.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:13:50.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:13:50.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:13:50.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:13:50.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 48.733824061999997, 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.406 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:50.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:13:50.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:13:50.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.412 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 48.733804835000001, 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:13:50.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.857731422999997, 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:50.417 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.418 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.418 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:13:50.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.418 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:13:50.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:13:50.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:13:50.423 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96967 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548794100999999, 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028701085000000001, 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.424 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96967 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848739+0000", 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848741+0000", 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.153e-06 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848743+0000", 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.723e-06 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848746+0000", 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.858e-06 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877369+0000", 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028623282999999999 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877406+0000", 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6139000000000001e-05 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877441+0000", 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151000000000003e-05 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96968 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:50.428 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548785540000001, 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028858604999999999, 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96968 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848749+0000", 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.091e-06 2024-09-13T07:13:50.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848752+0000", 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7769999999999998e-06 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848754+0000", 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2869999999999999e-06 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877468+0000", 2024-09-13T07:13:50.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028714185 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877541+0000", 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2927000000000002e-05 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877607+0000", 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5560000000000002e-05 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.432 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96969 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548777392000002, 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028944845, 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96969 2024-09-13T07:13:50.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.434 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848757+0000", 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848761+0000", 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848763+0000", 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.311e-06 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877630+0000", 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866326000000001 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877666+0000", 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6127000000000002e-05 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877701+0000", 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5413000000000001e-05 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.437 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96970 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548768401, 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029100663999999998, 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96970 2024-09-13T07:13:50.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:13:50.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848766+0000", 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.097e-06 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848769+0000", 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848772+0000", 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4269999999999998e-06 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877729+0000", 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028957488 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877800+0000", 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1113999999999997e-05 2024-09-13T07:13:50.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877866+0000", 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5882000000000004e-05 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96971 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548760270999999, 2024-09-13T07:13:50.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029181689, 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96971 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.443 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848774+0000", 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1209999999999999e-06 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848777+0000", 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.926e-06 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848780+0000", 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:13:50.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877884+0000", 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029104076999999999 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877919+0000", 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5655999999999999e-05 2024-09-13T07:13:50.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877955+0000", 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5782999999999999e-05 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96972 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.54875208, 2024-09-13T07:13:50.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029338843999999999, 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96972 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:50.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848781+0000", 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848783+0000", 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1659999999999999e-06 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848785+0000", 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.818e-06 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848788+0000", 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4569999999999999e-06 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877984+0000", 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029196203 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878055+0000", 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0622000000000002e-05 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878120+0000", 2024-09-13T07:13:50.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5816999999999997e-05 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96973 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548726909999999, 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029407188000000001, 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96973 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:13:50.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848806+0000", 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848808+0000", 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7039999999999999e-06 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848813+0000", 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6360000000000002e-06 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848815+0000", 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5799999999999999e-06 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878143+0000", 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029327678999999999 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878179+0000", 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5587000000000003e-05 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878214+0000", 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5456000000000001e-05 2024-09-13T07:13:50.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96974 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548715610999999, 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029557028999999999, 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.457 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96974 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848821+0000", 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5140000000000002e-06 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848825+0000", 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848827+0000", 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.39e-06 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878240+0000", 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029413174 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878310+0000", 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9813000000000004e-05 2024-09-13T07:13:50.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878375+0000", 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5122000000000004e-05 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96975 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548704569999998, 2024-09-13T07:13:50.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029640264, 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96975 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848830+0000", 2024-09-13T07:13:50.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.235e-06 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848833+0000", 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.796e-06 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848835+0000", 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.402e-06 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878397+0000", 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029561548999999999 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878434+0000", 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6721000000000003e-05 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878469+0000", 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5855999999999997e-05 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96976 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548696450000001, 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029795095000000001, 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.466 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96976 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848838+0000", 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.156e-06 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.468 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848841+0000", 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9110000000000002e-06 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848843+0000", 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878496+0000", 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029652371 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.469 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878567+0000", 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0784000000000005e-05 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878632+0000", 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5783000000000004e-05 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.470 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96977 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548688319, 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029869717, 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96977 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.471 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.472 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848846+0000", 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1000000000000001e-06 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848849+0000", 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.835e-06 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848851+0000", 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.473 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878643+0000", 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029791405 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878678+0000", 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5139999999999999e-05 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878715+0000", 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7119000000000002e-05 2024-09-13T07:13:50.474 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96978 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548680302000001, 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030029390999999999, 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96978 2024-09-13T07:13:50.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:13:50.476 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848854+0000", 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1039999999999999e-06 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848857+0000", 2024-09-13T07:13:50.477 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8399999999999999e-06 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848859+0000", 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878742+0000", 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029882140000000001 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878813+0000", 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1771000000000001e-05 2024-09-13T07:13:50.478 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878883+0000", 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9457999999999998e-05 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96979 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548672369999998, 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030106193999999999, 2024-09-13T07:13:50.479 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96979 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.481 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848862+0000", 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.234e-06 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848865+0000", 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.836e-06 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848868+0000", 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878895+0000", 2024-09-13T07:13:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030027642 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878931+0000", 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5624999999999999e-05 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878968+0000", 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6720000000000001e-05 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.483 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96980 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548664210999998, 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030267181000000001, 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.484 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96980 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.485 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848869+0000", 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.088e-06 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.486 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848873+0000", 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9019999999999999e-06 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848875+0000", 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1950000000000002e-06 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878996+0000", 2024-09-13T07:13:50.487 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03012068 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879067+0000", 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1100999999999996e-05 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879137+0000", 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9424999999999993e-05 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96981 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548650390999999, 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030344368, 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96981 2024-09-13T07:13:50.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:13:50.490 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848885+0000", 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848888+0000", 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.491 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848891+0000", 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.554e-06 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879159+0000", 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030268155000000001 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879192+0000", 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:13:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879228+0000", 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6146999999999998e-05 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96982 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 22.548641126, 2024-09-13T07:13:50.493 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030488562, 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96982 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.494 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848892+0000", 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.495 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848894+0000", 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2079999999999999e-06 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848896+0000", 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8679999999999999e-06 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848899+0000", 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.496 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879246+0000", 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030346838000000001 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879317+0000", 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1254000000000006e-05 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879381+0000", 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4331000000000005e-05 2024-09-13T07:13:50.497 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.498 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.498 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:50.499 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:50.500 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:50.500+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:50.600 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:50.601+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:50.642 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:50.642 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:50.642 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:50.642 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:50.650 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:50.734 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:50.735+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:50.792 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:50.792 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:50.792 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:50.793 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:50.793 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:50.801 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:50.802 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:50.802+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:50.902 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:50.903+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 72.908788677000004, 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.946 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:13:50.948 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.949 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:50.950 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.951 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.361879805000001, 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.952 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.953 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.954 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:13:50.955 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:13:50.956 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:13:50.957 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88519 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.360721971, 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017910447999999999, 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.958 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88519 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.959 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585257+0000", 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585267+0000", 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0635000000000001e-05 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585289+0000", 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1846999999999999e-05 2024-09-13T07:13:50.960 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585299+0000", 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4949999999999997e-06 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585347+0000", 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7672e-05 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.961 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585500+0000", 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015363700000000001 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.588596+0000", 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030953669999999999 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603068+0000", 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014472063 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.962 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603106+0000", 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8435e-05 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603142+0000", 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6488e-05 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603169+0000", 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6959000000000001e-05 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.963 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:89021 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 26.416245058000001, 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02102269, 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.964 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 89021 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.965 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529734+0000", 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529742+0000", 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8290000000000008e-06 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529757+0000", 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4868999999999999e-05 2024-09-13T07:13:50.966 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529763+0000", 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.93e-06 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529803+0000", 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0404999999999997e-05 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529809+0000", 2024-09-13T07:13:50.967 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531015+0000", 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012062049999999999 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531057+0000", 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1921000000000001e-05 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.968 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.534526+0000", 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034694909999999999 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550683+0000", 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016157169999999998 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550697+0000", 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3351e-05 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.969 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550731+0000", 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3997e-05 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550759+0000", 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7861e-05 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95026 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.970 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.250230991999999, 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014972104999999999, 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95026 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.971 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695752+0000", 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.283e-06 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.972 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695758+0000", 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695762+0000", 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710581+0000", 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014818934000000001 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.973 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710653+0000", 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2457000000000001e-05 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710722+0000", 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8633999999999994e-05 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95027 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.974 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.250214883000002, 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015079646, 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95027 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.975 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695768+0000", 2024-09-13T07:13:50.976 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695773+0000", 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.592e-06 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695778+0000", 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710749+0000", 2024-09-13T07:13:50.977 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014970809 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710805+0000", 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6149000000000001e-05 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710846+0000", 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0840999999999997e-05 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.978 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95028 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.250199187, 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015231216000000001, 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95028 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.979 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695781+0000", 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.980 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695783+0000", 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9250000000000002e-06 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695789+0000", 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.835e-06 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695793+0000", 2024-09-13T07:13:50.981 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3000000000000003e-06 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710881+0000", 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015087695999999999 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710947+0000", 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6169999999999998e-05 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.982 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.711013+0000", 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5643999999999997e-05 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95080 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248598059999999, 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014948011000000001, 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.983 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95080 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.984 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697385+0000", 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.187e-06 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.985 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697388+0000", 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9459999999999998e-06 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697390+0000", 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5600000000000001e-06 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712211+0000", 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014820224 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.986 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712276+0000", 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5650999999999999e-05 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712331+0000", 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4727e-05 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.987 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95081 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248588603999998, 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015124461000000001, 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95081 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697394+0000", 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.418e-06 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697397+0000", 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697399+0000", 2024-09-13T07:13:50.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.508e-06 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712367+0000", 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01496809 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712442+0000", 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5290999999999995e-05 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712517+0000", 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4396999999999999e-05 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95082 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248579920000001, 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015249593000000001, 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.992 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95082 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:50.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697402+0000", 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.311e-06 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697405+0000", 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:13:50.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697407+0000", 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712557+0000", 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015149207 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712609+0000", 2024-09-13T07:13:50.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2286e-05 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712651+0000", 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1687999999999998e-05 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95083 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3727360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248571429999998, 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015418032, 2024-09-13T07:13:50.996 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95083 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697409+0000", 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.049e-06 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697414+0000", 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.783e-06 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697417+0000", 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.587e-06 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712683+0000", 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015266083 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:50.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712759+0000", 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6627999999999999e-05 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712828+0000", 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8121999999999996e-05 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.000 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95084 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3776512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248562158999999, 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015560297000000001, 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95084 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697420+0000", 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697423+0000", 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3359999999999999e-06 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697425+0000", 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.452e-06 2024-09-13T07:13:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712864+0000", 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015438927999999999 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712938+0000", 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3458e-05 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712979+0000", 2024-09-13T07:13:51.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1251999999999997e-05 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95085 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248551369000001, 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015726161999999998, 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.005 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95085 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697429+0000", 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697431+0000", 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5230000000000001e-06 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697434+0000", 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:13:51.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697436+0000", 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3889999999999999e-06 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713008+0000", 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015571295000000001 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713084+0000", 2024-09-13T07:13:51.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6537000000000003e-05 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713156+0000", 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1616e-05 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95086 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3874816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:51.009 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248542363999999, 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015894697999999999, 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95086 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697438+0000", 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697440+0000", 2024-09-13T07:13:51.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.159e-06 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697443+0000", 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9359999999999999e-06 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697445+0000", 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713191+0000", 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015746343999999999 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713269+0000", 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7737000000000005e-05 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713333+0000", 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4233000000000007e-05 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.013 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95087 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248534025000001, 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01604939, 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95087 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.014 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697448+0000", 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.094e-06 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697451+0000", 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8949999999999998e-06 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697454+0000", 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.999e-06 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713361+0000", 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015907092000000001 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713433+0000", 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2606000000000002e-05 2024-09-13T07:13:51.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713496+0000", 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2908000000000001e-05 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95088 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248523885000001, 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016219289000000001, 2024-09-13T07:13:51.018 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95088 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697459+0000", 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.764e-06 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697462+0000", 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4240000000000002e-06 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697464+0000", 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3989999999999998e-06 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713532+0000", 2024-09-13T07:13:51.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016067403000000001 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713607+0000", 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5476999999999996e-05 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713676+0000", 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9115999999999994e-05 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95089 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248514031999999, 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016381144, 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95089 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697468+0000", 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.734e-06 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697474+0000", 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697479+0000", 2024-09-13T07:13:51.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713703+0000", 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016224643 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4828999999999998e-05 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713848+0000", 2024-09-13T07:13:51.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0029000000000003e-05 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95090 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248498233999999, 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016573104000000002, 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.027 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95090 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697482+0000", 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697484+0000", 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.916e-06 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697496+0000", 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697506+0000", 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0477e-05 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713903+0000", 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016396758000000001 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713981+0000", 2024-09-13T07:13:51.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7892999999999994e-05 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714056+0000", 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5214000000000002e-05 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95091 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248464812999998, 2024-09-13T07:13:51.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016674768, 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95091 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697518+0000", 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4789999999999999e-06 2024-09-13T07:13:51.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697527+0000", 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3799999999999994e-06 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697532+0000", 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714080+0000", 2024-09-13T07:13:51.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016547798999999998 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714135+0000", 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5294000000000003e-05 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714191+0000", 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5606999999999998e-05 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95092 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4169728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.248444025000001, 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017373829, 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95092 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697536+0000", 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697540+0000", 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4879999999999999e-06 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697548+0000", 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0139999999999998e-06 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697553+0000", 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2460000000000003e-06 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714754+0000", 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017200653 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714831+0000", 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7156000000000005e-05 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714911+0000", 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9740999999999994e-05 2024-09-13T07:13:51.039 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.040 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.040 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.040 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.040 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:51.041 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:51.042 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:51.042+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:51.142 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:51.143+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:51.189 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:51.190 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:51.190 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:13:51.190 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:51.201 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:51.242 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:51.243+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:51.344 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:51.345+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:51.349 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:51.349 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:51.349 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:51.350 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:51.350 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:51.358 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:51.445 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:51.446+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:51.502 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.148762302000002, 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99762887700000002, 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.503 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.504 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:13:51.505 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.506 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351641+0000", 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99754909000000003 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351653+0000", 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2248e-05 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.507 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88604 3.e 3:75ae40da:::rbd_header.117390733e87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.779931209000001, 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20280552800000001, 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88604 2024-09-13T07:13:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.509 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722855+0000", 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722859+0000", 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722870+0000", 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0434999999999999e-05 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722875+0000", 2024-09-13T07:13:51.510 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722912+0000", 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6943000000000002e-05 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722963+0000", 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1103000000000002e-05 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723070+0000", 2024-09-13T07:13:51.511 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106279 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723472+0000", 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040248200000000001 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925590+0000", 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20211748900000001 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.512 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925624+0000", 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4801000000000001e-05 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925661+0000", 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6643999999999997e-05 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88605 3.e 3:75ae40da:::rbd_header.117390733e87:head [watch ping cookie 139824472318400 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:51.513 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.729724141999998, 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15274379900000001, 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88605 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.514 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773062+0000", 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.515 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773066+0000", 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1570000000000002e-06 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773073+0000", 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5490000000000003e-06 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.516 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773080+0000", 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773125+0000", 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4849999999999999e-05 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for rw locks", 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773190+0000", 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5567000000000004e-05 2024-09-13T07:13:51.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925689+0000", 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15249922899999999 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925746+0000", 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6369999999999997e-05 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925806+0000", 2024-09-13T07:13:51.518 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0668999999999998e-05 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94703 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3366912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817606091999998, 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027375494, 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.519 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94703 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.520 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685180+0000", 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685183+0000", 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685189+0000", 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0730000000000001e-06 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.521 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685193+0000", 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712474+0000", 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027280787000000001 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712507+0000", 2024-09-13T07:13:51.522 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2786000000000001e-05 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712556+0000", 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9203000000000003e-05 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94704 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817588703999999, 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027564668000000001, 2024-09-13T07:13:51.523 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94704 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.524 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685200+0000", 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.525 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685207+0000", 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685212+0000", 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1449999999999999e-06 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712597+0000", 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027385158 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.526 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712681+0000", 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3961000000000005e-05 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712763+0000", 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1558999999999996e-05 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94705 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817558679000001, 2024-09-13T07:13:51.527 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027678611999999998, 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94705 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.528 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685230+0000", 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4660000000000002e-06 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685237+0000", 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685242+0000", 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8729999999999998e-06 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712793+0000", 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027551022000000001 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.530 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712835+0000", 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2537000000000003e-05 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712907+0000", 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1556000000000004e-05 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94706 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817541137999999, 2024-09-13T07:13:51.531 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027861107999999999, 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94706 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:51.532 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685245+0000", 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.533 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685247+0000", 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9479999999999998e-06 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685253+0000", 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685258+0000", 2024-09-13T07:13:51.534 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.549e-06 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712946+0000", 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027688252 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713033+0000", 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7472000000000005e-05 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.535 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713107+0000", 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3478000000000004e-05 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94707 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817525608, 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028004273, 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.536 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94707 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.537 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685263+0000", 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8449999999999999e-06 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.538 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685283+0000", 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0165000000000001e-05 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685289+0000", 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7189999999999998e-06 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713138+0000", 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027849056000000001 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.539 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713189+0000", 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1801000000000001e-05 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713265+0000", 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5983999999999996e-05 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.540 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94708 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.81749327, 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028161510000000001, 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94708 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.541 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.542 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685296+0000", 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-06 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685302+0000", 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685306+0000", 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.543 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713301+0000", 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027995176 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713372+0000", 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0870000000000004e-05 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713455+0000", 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2956999999999999e-05 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.544 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94709 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817476430999999, 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357679, 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.545 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94709 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.546 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685312+0000", 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685318+0000", 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.547 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685323+0000", 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713505+0000", 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028182713000000002 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713592+0000", 2024-09-13T07:13:51.548 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6397000000000006e-05 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713668+0000", 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6188999999999998e-05 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94710 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:51.549 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817458975000001, 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028518801, 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94710 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.550 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685327+0000", 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685330+0000", 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3120000000000001e-06 2024-09-13T07:13:51.551 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685337+0000", 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685342+0000", 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713700+0000", 2024-09-13T07:13:51.552 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357838999999999 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8438999999999999e-05 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713847+0000", 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8540000000000004e-05 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.553 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94711 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817440302000001, 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640149, 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94711 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.554 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.555 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685348+0000", 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685355+0000", 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685359+0000", 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-13T07:13:51.556 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713875+0000", 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028516414 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713931+0000", 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713987+0000", 2024-09-13T07:13:51.557 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6084e-05 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94712 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817423414, 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028779301, 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.558 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94712 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685365+0000", 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685371+0000", 2024-09-13T07:13:51.560 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7620000000000001e-06 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685375+0000", 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.42e-06 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714016+0000", 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640591 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714079+0000", 2024-09-13T07:13:51.561 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3288999999999997e-05 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714143+0000", 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3361999999999994e-05 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94713 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817407201000002, 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028911175000000001, 2024-09-13T07:13:51.562 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94713 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:51.563 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685379+0000", 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685381+0000", 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1509999999999998e-06 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.564 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685387+0000", 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685391+0000", 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5920000000000002e-06 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714169+0000", 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028777971999999999 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.565 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714235+0000", 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5782000000000001e-05 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714291+0000", 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5525000000000001e-05 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94714 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817366923000002, 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029006134999999999, 2024-09-13T07:13:51.566 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94714 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.567 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685419+0000", 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685430+0000", 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015e-05 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685441+0000", 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1049e-05 2024-09-13T07:13:51.568 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685446+0000", 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714313+0000", 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866640999999998 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714368+0000", 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5024e-05 2024-09-13T07:13:51.569 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7843000000000002e-05 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94715 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817335226000001, 2024-09-13T07:13:51.570 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029140899000000001, 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94715 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.571 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685451+0000", 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685454+0000", 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.627e-06 2024-09-13T07:13:51.572 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.573 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.573 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.573 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685466+0000", 2024-09-13T07:13:51.573 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2510000000000001e-05 2024-09-13T07:13:51.577 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.577 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.577 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.577 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685472+0000", 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714453+0000", 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028980599999999999 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714524+0000", 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1186999999999995e-05 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.578 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714592+0000", 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8668000000000001e-05 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94716 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817310314, 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029223545, 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.579 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94716 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:51.580 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685479+0000", 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4980000000000001e-06 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.581 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685485+0000", 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685489+0000", 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714617+0000", 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029128106000000001 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.582 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714662+0000", 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.456e-05 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714700+0000", 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8281000000000002e-05 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94717 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:51.583 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817293409000001, 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029398906999999998, 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94717 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.584 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685495+0000", 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0499999999999999e-06 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.585 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685501+0000", 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9569999999999999e-06 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685506+0000", 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3249999999999998e-06 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714726+0000", 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029219218000000002 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.586 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714804+0000", 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8610999999999997e-05 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714892+0000", 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8114000000000004e-05 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94718 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:51.587 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.81727613, 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029526177000000001, 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94718 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685510+0000", 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685513+0000", 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4360000000000001e-06 2024-09-13T07:13:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685519+0000", 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685523+0000", 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714918+0000", 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029395260999999999 2024-09-13T07:13:51.590 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714985+0000", 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6737999999999997e-05 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715037+0000", 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1712999999999998e-05 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94719 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 23.817259613000001, 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029657744, 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94719 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:51.592 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:51.593 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685529+0000", 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685535+0000", 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685539+0000", 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.594 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715062+0000", 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029522738 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715125+0000", 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3229000000000001e-05 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715185+0000", 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9441999999999998e-05 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.595 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.596 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:51.596 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:51.596 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:51.597 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:51.598+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:51.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:51.698 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:51.699+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:51.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:51.799 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:51.800+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:51.834 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:51.834 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:51.834 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:51.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:51.900 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:51.901+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:51.991 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51075 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456551034, 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081563839999999999, 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51075 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535172+0000", 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535178+0000", 2024-09-13T07:13:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535183+0000", 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000001e-06 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616726+0000", 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081543051000000005 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616734+0000", 2024-09-13T07:13:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51076 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456534034000001, 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081579246999999994, 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51076 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535189+0000", 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9709999999999998e-06 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535195+0000", 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0739999999999998e-06 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535199+0000", 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5549999999999996e-06 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616759+0000", 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081559926000000005 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616766+0000", 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51077 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456517415999997, 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081595972000000003, 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51077 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535203+0000", 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535210+0000", 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535217+0000", 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3490000000000003e-06 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535222+0000", 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616791+0000", 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081568733000000004 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616799+0000", 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.405e-06 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51078 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456493737999999, 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081603360999999999, 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51078 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535229+0000", 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535236+0000", 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.489e-06 2024-09-13T07:13:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535240+0000", 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616823+0000", 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081582882999999995 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616831+0000", 2024-09-13T07:13:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2280000000000001e-06 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51079 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456476285000001, 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081617985000000004, 2024-09-13T07:13:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51079 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535244+0000", 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535247+0000", 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535253+0000", 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2249999999999997e-06 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535258+0000", 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8899999999999998e-06 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616855+0000", 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081597307999999993 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616863+0000", 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6580000000000002e-06 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51080 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456458353999999, 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081642305999999998, 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51080 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535262+0000", 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535264+0000", 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535275+0000", 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0936e-05 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535281+0000", 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616889+0000", 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081608705000000004 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616905+0000", 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5398999999999999e-05 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51081 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456435366999997, 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081649656000000001, 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51081 2024-09-13T07:13:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535285+0000", 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535288+0000", 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535294+0000", 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-13T07:13:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535299+0000", 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616928+0000", 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081629060000000003 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616935+0000", 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.968e-06 2024-09-13T07:13:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51082 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456417356000003, 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081660766999999995, 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51082 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535303+0000", 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535305+0000", 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535312+0000", 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535317+0000", 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616957+0000", 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081640502000000004 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616964+0000", 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51083 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456399722, 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081673832000000002, 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51083 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535323+0000", 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535329+0000", 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2650000000000002e-06 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535334+0000", 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616988+0000", 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081653901000000001 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616995+0000", 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1980000000000004e-06 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51084 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456382398999999, 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081686317999999994, 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51084 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535338+0000", 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535340+0000", 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0640000000000001e-06 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535347+0000", 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535351+0000", 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617017+0000", 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081665777999999994 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617025+0000", 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5789999999999998e-06 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51085 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456365017000003, 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081702650000000002, 2024-09-13T07:13:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51085 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535358+0000", 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535364+0000", 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535369+0000", 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617051+0000", 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081681576000000006 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617059+0000", 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51086 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.456347223000002, 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081714234999999996, 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51086 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535373+0000", 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535376+0000", 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535382+0000", 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3899999999999998e-06 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535387+0000", 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7559999999999999e-06 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617081+0000", 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081694514999999995 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617088+0000", 2024-09-13T07:13:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.371254918999998, 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82619132900000003, 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620465+0000", 2024-09-13T07:13:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620483+0000", 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620497+0000", 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4248e-05 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620505+0000", 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9559999999999997e-06 2024-09-13T07:13:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620563+0000", 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7667999999999997e-05 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620573+0000", 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9289999999999993e-06 2024-09-13T07:13:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446642+0000", 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82606955599999998 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446657+0000", 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4932e-05 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.371189341000001, 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82648916299999997, 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620531+0000", 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620535+0000", 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3959999999999999e-06 2024-09-13T07:13:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620545+0000", 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5030000000000003e-06 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620551+0000", 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4690000000000002e-06 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620596+0000", 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4821000000000003e-05 2024-09-13T07:13:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620604+0000", 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6359999999999994e-06 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447008+0000", 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82640484000000003 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447021+0000", 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2320999999999999e-05 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.277472945, 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73282608599999999, 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714247+0000", 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714303+0000", 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5475e-05 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714338+0000", 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5611000000000002e-05 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714345+0000", 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714410+0000", 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5223999999999998e-05 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714417+0000", 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447059+0000", 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73264227900000001 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447074+0000", 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5078e-05 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.277295174999999, 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73271206499999997, 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714425+0000", 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714430+0000", 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714441+0000", 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0645999999999999e-05 2024-09-13T07:13:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714450+0000", 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714489+0000", 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8974000000000003e-05 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714494+0000", 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447125+0000", 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73263175700000005 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447138+0000", 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2532e-05 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.272151317999999, 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72759390000000002, 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719569+0000", 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719586+0000", 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7397999999999998e-05 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719597+0000", 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0368e-05 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719603+0000", 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719649+0000", 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5818000000000001e-05 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719658+0000", 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0680000000000003e-06 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447157+0000", 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72749929599999996 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447164+0000", 2024-09-13T07:13:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3659999999999997e-06 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.271949120999999, 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72744244800000002, 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719771+0000", 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719775+0000", 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719782+0000", 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3949999999999998e-06 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719789+0000", 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4719999999999998e-06 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719830+0000", 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1140000000000003e-05 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719841+0000", 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0808999999999999e-05 2024-09-13T07:13:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447202+0000", 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72736066499999996 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447214+0000", 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2472e-05 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.258643438, 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71610967800000003, 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733095+0000", 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733107+0000", 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1902999999999999e-05 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733115+0000", 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733154+0000", 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8501000000000003e-05 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733159+0000", 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2449999999999998e-06 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447255+0000", 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71409564299999995 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447366+0000", 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111273 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447510+0000", 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014420399999999999 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448696+0000", 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001186246 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449068+0000", 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037139399999999999 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449145+0000", 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449187+0000", 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2407999999999998e-05 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [write 0~0] snapc 22=[22] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.258487656, 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71614144199999996, 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733232+0000", 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733237+0000", 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2880000000000003e-06 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733249+0000", 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2259000000000001e-05 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733256+0000", 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.742e-06 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733284+0000", 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8708e-05 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733294+0000", 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4089999999999992e-06 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447905+0000", 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71461088299999997 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448019+0000", 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011461800000000001 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448195+0000", 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017585699999999999 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448989+0000", 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079339899999999997 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449320+0000", 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000331825 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449350+0000", 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0111999999999999e-05 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449375+0000", 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4278e-05 2024-09-13T07:13:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:52.082 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:52.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:52.084 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:52.084+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:52.128 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:52.128 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:52.128 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-13T07:13:52.128 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:13:52.128 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:13:52.128 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-13T07:13:52.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:52.184 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:52.185+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:52.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:13:52.240 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:52.241 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:52.241 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:52.241 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:52.243 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:52.243 INFO:teuthology.orchestra.run.smithi184.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:13:52.243 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:52.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:52.291 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:52.292+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:52.306 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:52.325 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:52.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:52.348 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:52.366 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:52.366 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:52.386 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:52.388+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:52.398 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:52.398 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:52.430 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:52.433 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:52.433 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:52.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:52.452 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:52.453 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:52.469 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:52.487 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:52.487+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:52.559 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:52.560 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:52.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:52.587 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:52.589+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:52.675 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:52.676 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:52.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:52.728 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:52.729+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:52.789 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:52.789+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:52.791 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:52.791 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:52.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:52.889 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:52.890+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:52.947 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:52.947 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:52.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:53.000 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:53.001+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:53.062 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:53.062 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:53.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:53.091 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:53.091+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:13:53.203 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:53.204+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:53.233 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:53.233 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:53.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:53.327 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:53.328+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:53.390 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:53.390 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:53.394 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:53.395+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:53.399 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:53.495 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:53.495+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:53.502 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:53.502 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:53.511 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:53.596 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:53.596+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:53.654 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:53.654 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:53.663 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:53.696 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:53.698+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:53.797 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:53.797+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:53.805 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:53.805 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:53.814 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:53.898 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:53.898+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:53.957 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:53.957 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:53.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:53.998 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:54.000+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:54.100 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:54.100+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:54.109 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:54.109 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:54.118 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:54.201 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:54.202+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:54.225 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:54.225 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:54.235 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:54.302 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:54.303+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:54.346 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:54.355 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:54.403 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:54.403+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:54.457 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:54.457 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:54.466 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:54.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:54.507 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:54.507+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:54.605 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:54.605+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:54.610 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:54.610 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:54.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:54.620 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:54.620 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:54.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:54.637 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:54.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:54.669 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:54.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:54.687 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:54.703 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:54.705 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:54.705+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:54.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:54.720 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:54.723 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:54.723 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:54.733 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:54.736 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:54.736 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:54.807 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:54.808+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:13:54.875 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:54.884 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:54.928 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:54.928+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:54.987 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:13:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:13:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:13:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:13:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:13:54.987 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.633238542000001, 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99762887700000002, 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:54.988 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:54.989 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351641+0000", 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99754909000000003 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351653+0000", 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2248e-05 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88604 3.e 3:75ae40da:::rbd_header.117390733e87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.264407449, 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20280552800000001, 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:54.992 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88604 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:54.993 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722855+0000", 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722859+0000", 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722870+0000", 2024-09-13T07:13:54.994 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0434999999999999e-05 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722875+0000", 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722912+0000", 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6943000000000002e-05 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722963+0000", 2024-09-13T07:13:54.995 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1103000000000002e-05 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723070+0000", 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106279 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723472+0000", 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040248200000000001 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925590+0000", 2024-09-13T07:13:54.996 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20211748900000001 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925624+0000", 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4801000000000001e-05 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925661+0000", 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6643999999999997e-05 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.997 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88605 3.e 3:75ae40da:::rbd_header.117390733e87:head [watch ping cookie 139824472318400 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 31.214200382000001, 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15274379900000001, 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88605 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.998 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773062+0000", 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:54.999 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773066+0000", 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1570000000000002e-06 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773073+0000", 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5490000000000003e-06 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773080+0000", 2024-09-13T07:13:55.000 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773125+0000", 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4849999999999999e-05 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for rw locks", 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773190+0000", 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5567000000000004e-05 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.001 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925689+0000", 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15249922899999999 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925746+0000", 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6369999999999997e-05 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925806+0000", 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0668999999999998e-05 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.002 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94703 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3366912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.302082332000001, 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027375494, 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94703 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.003 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.004 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685180+0000", 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685183+0000", 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685189+0000", 2024-09-13T07:13:55.005 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0730000000000001e-06 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685193+0000", 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712474+0000", 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027280787000000001 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.006 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712507+0000", 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2786000000000001e-05 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712556+0000", 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9203000000000003e-05 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94704 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:55.007 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.302064944000001, 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027564668000000001, 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94704 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.008 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685200+0000", 2024-09-13T07:13:55.009 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685207+0000", 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685212+0000", 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1449999999999999e-06 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712597+0000", 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027385158 2024-09-13T07:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712681+0000", 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3961000000000005e-05 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712763+0000", 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1558999999999996e-05 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94705 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.011 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.302034919, 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027678611999999998, 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94705 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:55.012 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.013 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685230+0000", 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4660000000000002e-06 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685237+0000", 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685242+0000", 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8729999999999998e-06 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.014 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712793+0000", 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027551022000000001 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712835+0000", 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2537000000000003e-05 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712907+0000", 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1556000000000004e-05 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94706 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.302017377999999, 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027861107999999999, 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94706 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.016 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685245+0000", 2024-09-13T07:13:55.017 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685247+0000", 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9479999999999998e-06 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685253+0000", 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.018 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685258+0000", 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.549e-06 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712946+0000", 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027688252 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713033+0000", 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7472000000000005e-05 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.019 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713107+0000", 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3478000000000004e-05 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94707 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.302001848, 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028004273, 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.020 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94707 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.021 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685263+0000", 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8449999999999999e-06 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685283+0000", 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0165000000000001e-05 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.022 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685289+0000", 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7189999999999998e-06 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713138+0000", 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027849056000000001 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.023 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713189+0000", 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1801000000000001e-05 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713265+0000", 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5983999999999996e-05 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94708 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.024 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301969509999999, 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028161510000000001, 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94708 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685296+0000", 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-06 2024-09-13T07:13:55.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685302+0000", 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685306+0000", 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713301+0000", 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027995176 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.027 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713372+0000", 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0870000000000004e-05 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713455+0000", 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2956999999999999e-05 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94709 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.028 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301952670999999, 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357679, 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94709 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685312+0000", 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:13:55.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685318+0000", 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685323+0000", 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713505+0000", 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028182713000000002 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.031 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713592+0000", 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6397000000000006e-05 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713668+0000", 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6188999999999998e-05 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.032 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94710 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301935215, 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028518801, 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94710 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.033 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.034 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685327+0000", 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685330+0000", 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3120000000000001e-06 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.035 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685337+0000", 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685342+0000", 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713700+0000", 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357838999999999 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.036 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8438999999999999e-05 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713847+0000", 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8540000000000004e-05 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.037 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94711 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301916542000001, 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640149, 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94711 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.038 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.039 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685348+0000", 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685355+0000", 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685359+0000", 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.040 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713875+0000", 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028516414 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713931+0000", 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713987+0000", 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6084e-05 2024-09-13T07:13:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94712 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301899654, 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028779301, 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.042 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94712 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.043 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685365+0000", 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.044 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685371+0000", 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7620000000000001e-06 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685375+0000", 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.42e-06 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714016+0000", 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640591 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.045 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714079+0000", 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3288999999999997e-05 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714143+0000", 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3361999999999994e-05 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94713 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301883441000001, 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028911175000000001, 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.047 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94713 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.048 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685379+0000", 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685381+0000", 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1509999999999998e-06 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.049 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685387+0000", 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685391+0000", 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5920000000000002e-06 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714169+0000", 2024-09-13T07:13:55.050 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028777971999999999 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714235+0000", 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5782000000000001e-05 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714291+0000", 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5525000000000001e-05 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.051 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94714 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301843163000001, 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029006134999999999, 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94714 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.052 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.053 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.057 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685419+0000", 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.058 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685430+0000", 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015e-05 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685441+0000", 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1049e-05 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.059 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685446+0000", 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714313+0000", 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866640999999998 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714368+0000", 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5024e-05 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.060 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7843000000000002e-05 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94715 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301811466, 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029140899000000001, 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.061 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94715 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.062 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685451+0000", 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685454+0000", 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.627e-06 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.063 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685466+0000", 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2510000000000001e-05 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685472+0000", 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714453+0000", 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028980599999999999 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.064 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714524+0000", 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1186999999999995e-05 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714592+0000", 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8668000000000001e-05 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94716 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.065 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301786554, 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029223545, 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94716 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.066 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685479+0000", 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4980000000000001e-06 2024-09-13T07:13:55.067 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685485+0000", 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685489+0000", 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714617+0000", 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029128106000000001 2024-09-13T07:13:55.068 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714662+0000", 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.456e-05 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714700+0000", 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8281000000000002e-05 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.069 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94717 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301769649000001, 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029398906999999998, 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94717 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.070 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.071 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685495+0000", 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0499999999999999e-06 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685501+0000", 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9569999999999999e-06 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.072 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685506+0000", 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3249999999999998e-06 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714726+0000", 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029219218000000002 2024-09-13T07:13:55.073 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714804+0000", 2024-09-13T07:13:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8610999999999997e-05 2024-09-13T07:13:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.074 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714892+0000", 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8114000000000004e-05 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94718 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301752369999999, 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029526177000000001, 2024-09-13T07:13:55.075 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94718 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.076 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685510+0000", 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685513+0000", 2024-09-13T07:13:55.077 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4360000000000001e-06 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685519+0000", 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685523+0000", 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:13:55.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714918+0000", 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029395260999999999 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714985+0000", 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6737999999999997e-05 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.079 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715037+0000", 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1712999999999998e-05 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94719 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 27.301735853, 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029657744, 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:13:55.080 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94719 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.081 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685529+0000", 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.082 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685535+0000", 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685539+0000", 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715062+0000", 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029522738 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.083 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715125+0000", 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3229000000000001e-05 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715185+0000", 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9441999999999998e-05 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:13:55.084 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:13:55.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:55.087 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:55.087+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:55.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:13:55.200 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:55.200 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:55.200 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:55.200 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:55.201 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:55.201+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:55.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:55.287 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:55.287+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:55.371 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:55.371 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:55.371 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:55.371 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:55.371 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:55.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:55.424 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:55.425+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51075 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953606803, 2024-09-13T07:13:55.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081563839999999999, 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51075 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535170+0000", 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535172+0000", 2024-09-13T07:13:55.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535178+0000", 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535183+0000", 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000001e-06 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616726+0000", 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081543051000000005 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616734+0000", 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51076 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953589803, 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081579246999999994, 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51076 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535187+0000", 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535189+0000", 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9709999999999998e-06 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535195+0000", 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0739999999999998e-06 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535199+0000", 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5549999999999996e-06 2024-09-13T07:13:55.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616759+0000", 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081559926000000005 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616766+0000", 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51077 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953573185000003, 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081595972000000003, 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51077 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535204+0000", 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535203+0000", 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535210+0000", 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535217+0000", 2024-09-13T07:13:55.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3490000000000003e-06 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535222+0000", 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616791+0000", 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081568733000000004 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616799+0000", 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.405e-06 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51078 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953549506999998, 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081603360999999999, 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51078 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:55.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535227+0000", 2024-09-13T07:13:55.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535229+0000", 2024-09-13T07:13:55.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-13T07:13:55.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535236+0000", 2024-09-13T07:13:55.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.489e-06 2024-09-13T07:13:55.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535240+0000", 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616823+0000", 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081582882999999995 2024-09-13T07:13:55.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616831+0000", 2024-09-13T07:13:55.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2280000000000001e-06 2024-09-13T07:13:55.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51079 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953532054, 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081617985000000004, 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51079 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535245+0000", 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535244+0000", 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535247+0000", 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535253+0000", 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2249999999999997e-06 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535258+0000", 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8899999999999998e-06 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616855+0000", 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081597307999999993 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616863+0000", 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6580000000000002e-06 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51080 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953514122999998, 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081642305999999998, 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51080 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535263+0000", 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535262+0000", 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535264+0000", 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535275+0000", 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0936e-05 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535281+0000", 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616889+0000", 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081608705000000004 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616905+0000", 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5398999999999999e-05 2024-09-13T07:13:55.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51081 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953491135999997, 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081649656000000001, 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51081 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535286+0000", 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535285+0000", 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535288+0000", 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4030000000000001e-06 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535294+0000", 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535299+0000", 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616928+0000", 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081629060000000003 2024-09-13T07:13:55.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616935+0000", 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.968e-06 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51082 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:55.521 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953473125000002, 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081660766999999995, 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51082 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535304+0000", 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535303+0000", 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535305+0000", 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535312+0000", 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535317+0000", 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616957+0000", 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081640502000000004 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616964+0000", 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51083 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953455491, 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081673832000000002, 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51083 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535321+0000", 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535323+0000", 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535329+0000", 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2650000000000002e-06 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535334+0000", 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616988+0000", 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081653901000000001 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.616995+0000", 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1980000000000004e-06 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51084 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953438167999998, 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081686317999999994, 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51084 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535339+0000", 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535338+0000", 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535340+0000", 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0640000000000001e-06 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535347+0000", 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535351+0000", 2024-09-13T07:13:55.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617017+0000", 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081665777999999994 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617025+0000", 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5789999999999998e-06 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51085 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953420786000002, 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081702650000000002, 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51085 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535356+0000", 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535358+0000", 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535364+0000", 2024-09-13T07:13:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535369+0000", 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617051+0000", 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081681576000000006 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617059+0000", 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51086 3.1f 3.c6a12dbf (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.953402992000001, 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081714234999999996, 2024-09-13T07:13:55.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51086 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535374+0000", 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535373+0000", 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535376+0000", 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-13T07:13:55.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535382+0000", 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3899999999999998e-06 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.535387+0000", 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7559999999999999e-06 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617081+0000", 2024-09-13T07:13:55.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.081694514999999995 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.617088+0000", 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.868310688000001, 2024-09-13T07:13:55.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82619132900000003, 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620466+0000", 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620465+0000", 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620483+0000", 2024-09-13T07:13:55.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620497+0000", 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4248e-05 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620505+0000", 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9559999999999997e-06 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620563+0000", 2024-09-13T07:13:55.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7667999999999997e-05 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620573+0000", 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9289999999999993e-06 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446642+0000", 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82606955599999998 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.446657+0000", 2024-09-13T07:13:55.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4932e-05 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.86824511, 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82648916299999997, 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620532+0000", 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620531+0000", 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620535+0000", 2024-09-13T07:13:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3959999999999999e-06 2024-09-13T07:13:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620545+0000", 2024-09-13T07:13:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5030000000000003e-06 2024-09-13T07:13:55.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620551+0000", 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4690000000000002e-06 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620596+0000", 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4821000000000003e-05 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.620604+0000", 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6359999999999994e-06 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447008+0000", 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.82640484000000003 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447021+0000", 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2320999999999999e-05 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.774528713999999, 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73282608599999999, 2024-09-13T07:13:55.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714248+0000", 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714247+0000", 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714303+0000", 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5475e-05 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714338+0000", 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5611000000000002e-05 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714345+0000", 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714410+0000", 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5223999999999998e-05 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714417+0000", 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447059+0000", 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73264227900000001 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447074+0000", 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5078e-05 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.774350943999998, 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73271206499999997, 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714425+0000", 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714430+0000", 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714441+0000", 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0645999999999999e-05 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714450+0000", 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:13:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714489+0000", 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8974000000000003e-05 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.714494+0000", 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447125+0000", 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.73263175700000005 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447138+0000", 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2532e-05 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.769207087000002, 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72759390000000002, 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719570+0000", 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719569+0000", 2024-09-13T07:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719586+0000", 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7397999999999998e-05 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719597+0000", 2024-09-13T07:13:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0368e-05 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719603+0000", 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719649+0000", 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5818000000000001e-05 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719658+0000", 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0680000000000003e-06 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447157+0000", 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72749929599999996 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447164+0000", 2024-09-13T07:13:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3659999999999997e-06 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3.400bb568 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.769004890000001, 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72744244800000002, 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719772+0000", 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719771+0000", 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719775+0000", 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719782+0000", 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3949999999999998e-06 2024-09-13T07:13:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719789+0000", 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4719999999999998e-06 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719830+0000", 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1140000000000003e-05 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.719841+0000", 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0808999999999999e-05 2024-09-13T07:13:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447202+0000", 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.72736066499999996 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447214+0000", 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2472e-05 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93671 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.755699206999999, 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71610967800000003, 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93671 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733077+0000", 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733095+0000", 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733107+0000", 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1902999999999999e-05 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733115+0000", 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-09-13T07:13:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733154+0000", 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8501000000000003e-05 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733159+0000", 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2449999999999998e-06 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447255+0000", 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71409564299999995 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447366+0000", 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111273 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447510+0000", 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014420399999999999 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448696+0000", 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001186246 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449068+0000", 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037139399999999999 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449145+0000", 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449187+0000", 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2407999999999998e-05 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:93672 3.8 3:16add002:::rbd_data.11738a001609.000000000000002c:head [write 0~0] snapc 22=[22] RETRY=3 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.755543424999999, 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71614144199999996, 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93672 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733233+0000", 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733232+0000", 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733237+0000", 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2880000000000003e-06 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733249+0000", 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2259000000000001e-05 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733256+0000", 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.742e-06 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733284+0000", 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8708e-05 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T07:13:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:27.733294+0000", 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4089999999999992e-06 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.447905+0000", 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71461088299999997 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448019+0000", 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011461800000000001 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448195+0000", 2024-09-13T07:13:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017585699999999999 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.448989+0000", 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079339899999999997 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449320+0000", 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000331825 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449350+0000", 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0111999999999999e-05 2024-09-13T07:13:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.449375+0000", 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4278e-05 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:55.587 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:55.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:55.589 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:55.589+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:55.689 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:55.689+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:55.736 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:55.736 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:55.736 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:55.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:55.790 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:55.791+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:55.851 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:55.851 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:55.851 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:55.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:55.901 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:55.902+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:55.992 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:55.993+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49523 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2699264~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.566177132999997, 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056285390999999997, 2024-09-13T07:13:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49523 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442063+0000", 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9379999999999999e-06 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442068+0000", 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442073+0000", 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:13:56.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497684+0000", 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055610930000000003 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497759+0000", 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5199999999999998e-05 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.498346+0000", 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058719100000000004 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49609 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 4075520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.548112801999999, 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055911012000000003, 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49609 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:56.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460127+0000", 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1430000000000001e-06 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460133+0000", 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5729999999999998e-06 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460138+0000", 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515783+0000", 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055645873999999998 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515861+0000", 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7979000000000007e-05 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.516036+0000", 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017504900000000001 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49618 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 999424~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.545598394000002, 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055816179, 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49618 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462639+0000", 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462642+0000", 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462647+0000", 2024-09-13T07:13:56.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462652+0000", 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518291+0000", 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055639051000000002 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518368+0000", 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6913e-05 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518456+0000", 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8219000000000004e-05 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49619 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.545581968999997, 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055989537999999998, 2024-09-13T07:13:56.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49619 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:56.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462659+0000", 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462665+0000", 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462669+0000", 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518483+0000", 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055813437 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518559+0000", 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5981000000000003e-05 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518646+0000", 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7175999999999993e-05 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49620 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1048576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.545564063999997, 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056152684000000001, 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49620 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:56.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462676+0000", 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462683+0000", 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:56.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462688+0000", 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518672+0000", 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055983988999999998 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518747+0000", 2024-09-13T07:13:56.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4633000000000002e-05 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518827+0000", 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000002e-05 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49621 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1073152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.545545316000002, 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056323023999999999, 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49621 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:56.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462695+0000", 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462701+0000", 2024-09-13T07:13:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462706+0000", 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6530000000000002e-06 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518854+0000", 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056147858000000002 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518934+0000", 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0813e-05 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519016+0000", 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1434999999999995e-05 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49622 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1097728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.54552803, 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056490088000000001, 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49622 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:56.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462712+0000", 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462718+0000", 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462722+0000", 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519042+0000", 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056320041000000001 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519120+0000", 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7607000000000004e-05 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519200+0000", 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166999999999993e-05 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49640 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1318912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.541723327, 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056480083, 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49640 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466514+0000", 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466517+0000", 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466524+0000", 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466529+0000", 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520731+0000", 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054202469000000003 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520808+0000", 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6854000000000007e-05 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522995+0000", 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00218681 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49653 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1490944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.541602115000003, 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055956026999999998, 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49653 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466638+0000", 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466643+0000", 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466648+0000", 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-13T07:13:56.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522424+0000", 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055775681000000001 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522499+0000", 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5828000000000007e-05 2024-09-13T07:13:56.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522592+0000", 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2781000000000005e-05 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49674 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1662976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.540188725999997, 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056005177000000003, 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49674 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468052+0000", 2024-09-13T07:13:56.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.497e-06 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468058+0000", 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2430000000000002e-06 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468063+0000", 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523872+0000", 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055808706999999999 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523960+0000", 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8919999999999996e-05 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.524055+0000", 2024-09-13T07:13:56.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4222999999999996e-05 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49695 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.537903065000002, 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056046466000000003, 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49695 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470338+0000", 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470345+0000", 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4100000000000002e-06 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470351+0000", 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525756+0000", 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055404954999999999 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525819+0000", 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2779000000000003e-05 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526382+0000", 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056291800000000003 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49696 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.537882367000002, 2024-09-13T07:13:56.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056222355000000002, 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49696 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470355+0000", 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470358+0000", 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470364+0000", 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470369+0000", 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526411+0000", 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056041476999999999 2024-09-13T07:13:56.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526489+0000", 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7787000000000006e-05 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526578+0000", 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9606999999999998e-05 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49704 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2056192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.536256516000002, 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055974318000000002, 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49704 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471981+0000", 2024-09-13T07:13:56.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471984+0000", 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471990+0000", 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7229999999999998e-06 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471995+0000", 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:56.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527788+0000", 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055793091000000003 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527861+0000", 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3038000000000001e-05 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527956+0000", 2024-09-13T07:13:56.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4536000000000004e-05 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49706 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2080768~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.536238709000003, 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056135461999999997, 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49706 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472006+0000", 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-13T07:13:56.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472012+0000", 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472017+0000", 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527980+0000", 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055963185999999998 2024-09-13T07:13:56.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528050+0000", 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979000000000002e-05 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528135+0000", 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4594999999999999e-05 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49708 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2105344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.536216547999999, 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056284837999999997, 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49708 2024-09-13T07:13:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472021+0000", 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472024+0000", 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.261e-06 2024-09-13T07:13:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472029+0000", 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472034+0000", 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0069999999999997e-06 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528161+0000", 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056126285999999997 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528232+0000", 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1000000000000005e-05 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528306+0000", 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4876000000000006e-05 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49711 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2129920~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.536199033999999, 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444877999999997, 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49711 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472041+0000", 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472047+0000", 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472052+0000", 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528331+0000", 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056278504 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528399+0000", 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7941000000000006e-05 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528484+0000", 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5249999999999999e-05 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49713 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2154496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.536027720999996, 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444079000000001, 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49713 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472214+0000", 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:13:56.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472228+0000", 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3964999999999999e-05 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472236+0000", 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4109999999999999e-06 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528509+0000", 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056273532000000001 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528581+0000", 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1338e-05 2024-09-13T07:13:56.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528655+0000", 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3858999999999999e-05 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49728 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2326528~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.53415382, 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055971704999999997, 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49728 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474087+0000", 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474093+0000", 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474098+0000", 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529880+0000", 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055782102 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529967+0000", 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6853000000000003e-05 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.530056+0000", 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9430000000000003e-05 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49767 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2695168~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.530064144000001, 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062028049000000002, 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49767 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478176+0000", 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478182+0000", 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478187+0000", 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-13T07:13:56.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532704+0000", 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054517200000000002 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532779+0000", 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5244e-05 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.540202+0000", 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074227429999999999 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:50078 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.516261964000002, 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055871918999999999, 2024-09-13T07:13:56.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50078 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491979+0000", 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-13T07:13:56.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491986+0000", 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491991+0000", 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544160+0000", 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052168953999999997 2024-09-13T07:13:56.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544241+0000", 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1213999999999999e-05 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.547848+0000", 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003606663 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.107 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:56.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:56.109 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:56.109+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:56.209 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:56.209+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:56.210 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:13:56.267 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:56.267 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:56.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:56.320 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:56.321+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:56.390 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:56.391 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:56.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:56.441 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:56.442+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:56.513 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:56.513+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.251115291999994, 2024-09-13T07:13:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.249637266999997, 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.246820217999996, 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.677391258, 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781567030999994, 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.734219737999993, 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.700721208999994, 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 67.112805066999996, 2024-09-13T07:13:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.842243199999999, 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:64537 3.9 e47/19 3:97a74db9:::rbd_data.117332306d66.0000000000000002:head v 47'372, mlcod=45'370)", 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.929629767999998, 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092884539999999998, 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.624988+0000", 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055812279999999997 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625018+0000", 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9496999999999999e-05 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625031+0000", 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625078+0000", 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7290000000000003e-05 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625096+0000", 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8091999999999999e-05 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628662+0000", 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035652449999999999 2024-09-13T07:13:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628696+0000", 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4035999999999997e-05 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:97494 3.0 e67/19 3:02ad2de2:::rbd_data.117387780896.0000000000000002:head v 67'375, mlcod=66'372)", 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.675428395000001, 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005628437, 2024-09-13T07:13:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873608+0000", 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875823+0000", 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022143660000000002 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875843+0000", 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9864000000000002e-05 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875854+0000", 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1345999999999999e-05 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875918+0000", 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4151000000000003e-05 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875939+0000", 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1016999999999999e-05 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879194+0000", 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032546820000000001 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879237+0000", 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3353000000000003e-05 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1145 3.4 e76/19 3:27c86583:::rbd_data.11738a001609.0000000000000046:22 v 76'478, mlcod=71'472)", 2024-09-13T07:13:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.033429734999999, 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063983709999999999, 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515618+0000", 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0787e-05 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515629+0000", 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0905e-05 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515634+0000", 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1710000000000001e-06 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.516333+0000", 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00069910600000000001 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.516351+0000", 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7822999999999999e-05 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.521968+0000", 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056176519999999999 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.522006+0000", 2024-09-13T07:13:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7246000000000002e-05 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1172 3.0 e84/19 3:075a22d2:::rbd_data.1173d7ab5a36.000000000000000f:1a v 84'423, mlcod=80'417)", 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.153288+0000", 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.395749460999999, 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033929540000000001, 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153288+0000", 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153288+0000", 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153287+0000", 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153292+0000", 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3800000000000002e-06 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153300+0000", 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.841e-06 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153307+0000", 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153372+0000", 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5679999999999995e-05 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153400+0000", 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7545000000000001e-05 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156642+0000", 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032426759999999999 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156681+0000", 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8112999999999999e-05 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.4.0:2822 3.9 e84/19 3:91ae4f65:::rbd_data.1173d7ab5a36.0000000000000022:1a v 84'496, mlcod=82'492)", 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.393628640999999, 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044564410000000002, 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155413+0000", 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0359999999999997e-06 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155427+0000", 2024-09-13T07:13:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3675e-05 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155435+0000", 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8739999999999995e-06 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155490+0000", 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155507+0000", 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7204999999999999e-05 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.159833+0000", 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043255990000000003 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.159865+0000", 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2325e-05 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1173 3.4 e84/19 3:2217e031:::rbd_data.1173d7ab5a36.0000000000000036:1a v 84'488, mlcod=82'484)", 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.155592+0000", 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.393445143999999, 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069439200000000001, 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155592+0000", 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155592+0000", 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155591+0000", 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155607+0000", 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5556000000000001e-05 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155618+0000", 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0842999999999999e-05 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155623+0000", 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0490000000000003e-06 2024-09-13T07:13:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155663+0000", 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0085e-05 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155685+0000", 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2693000000000001e-05 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162499+0000", 2024-09-13T07:13:56.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068136239999999999 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162536+0000", 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6789000000000003e-05 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.4.0:2824 3.9 e84/19 3:95c5389a:::rbd_data.1173d7ab5a36.000000000000000c:1a v 84'498, mlcod=82'492)", 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.155976+0000", 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.393061445000001, 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064142050000000001, 2024-09-13T07:13:56.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155976+0000", 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155976+0000", 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155975+0000", 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155980+0000", 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155990+0000", 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2690000000000005e-06 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155994+0000", 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5639999999999998e-06 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156029+0000", 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4696999999999997e-05 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156042+0000", 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869e-05 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162346+0000", 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063042840000000003 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162390+0000", 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3649999999999997e-05 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1174 3.4 e84/19 3:25e9d115:::rbd_data.1173d7ab5a36.000000000000000d:1a v 84'490, mlcod=82'484)", 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.156226+0000", 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.39281137, 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089488970000000008, 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156226+0000", 2024-09-13T07:13:56.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156226+0000", 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156225+0000", 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156231+0000", 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3199999999999996e-06 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156243+0000", 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1294000000000001e-05 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156249+0000", 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6560000000000003e-06 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.157990+0000", 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017409669999999999 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.158010+0000", 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0086999999999999e-05 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.165137+0000", 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071266180000000004 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.165175+0000", 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7642999999999999e-05 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1207 3.4 e89/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 v 89'516, mlcod=86'513)", 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:51.180199+0000", 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.368838277, 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033445800000000002, 2024-09-13T07:13:56.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180199+0000", 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180199+0000", 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180198+0000", 2024-09-13T07:13:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180207+0000", 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7010000000000002e-06 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180218+0000", 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.136e-05 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180226+0000", 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2300000000000008e-06 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180287+0000", 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0825000000000001e-05 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180308+0000", 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0727e-05 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.183495+0000", 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031865639999999998 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.183543+0000", 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8798000000000003e-05 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:107741 3.9 e94/19 3:95c0f194:::rbd_data.1173f6773d36.000000000000000d:head v 94'501, mlcod=85'499)", 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.26183694400000002, 2024-09-13T07:13:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038223659999999998, 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287204+0000", 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7520000000000004e-06 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287215+0000", 2024-09-13T07:13:56.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287225+0000", 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3689999999999996e-06 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287265+0000", 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0309999999999999e-05 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287278+0000", 2024-09-13T07:13:56.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2874000000000001e-05 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290981+0000", 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037032480000000001 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.291022+0000", 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1519e-05 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:107747 3.0 e94/19 3:07d9a7ed:::rbd_data.1173f6773d36.0000000000000019:head v 94'440, mlcod=91'437)", 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:56.290516+0000", 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.25852109499999998, 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033261160000000001, 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290516+0000", 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290516+0000", 2024-09-13T07:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290515+0000", 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290521+0000", 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1850000000000001e-06 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290533+0000", 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1364e-05 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290540+0000", 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.52e-06 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290589+0000", 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-05 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290611+0000", 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2526000000000001e-05 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.293805+0000", 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003193605 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.293842+0000", 2024-09-13T07:13:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7347e-05 2024-09-13T07:13:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:56.640 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:56.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:13:56.642 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:56.643+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:56.742 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:56.743+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:56.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:13:56.790 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:56.790 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:56.791 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:56.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:13:56.843 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:56.843+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:56.872 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:56.872 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:56.890 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:56.906 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:56.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:56.922 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:56.944 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:56.944+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:56.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:56.967 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:56.967 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:56.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:13:56.982 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:56.999 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:56.999 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:57.016 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:57.069 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:57.070+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:57.089 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:57.089 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:57.097 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:13:57.146 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:57.146+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:57.216 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:57.216 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:57.229 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:13:57.253 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:13:57.253 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:13:57.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:13:57.272 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:57.272+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:57.344 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:57.344 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:57.348 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:57.347+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:57.353 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:13:57.448 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:57.448+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:57.496 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:57.496 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:57.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:13:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:13:57.545 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:13:57.549 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:57.550+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:57.555 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-13T07:13:57.214988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":358285734,"num_objects":893,"num_object_clones":96,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":554,"num_read":144604,"num_read_kb":624711,"num_write":19331,"num_write_kb":358804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15850,"ondisk_log_size":15850,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":362592,"kb_used_data":20820,"kb_used_omap":20,"kb_used_meta":341483,"kb_avail":754612128,"statfs":{"total":773094113280,"available":772722819072,"internally_reserved":0,"allocated":21319680,"data_stored":77400875,"data_compressed":3117645,"data_compressed_allocated":5795840,"data_compressed_original":67697842,"omap_allocated":21248,"internal_metadata":349678848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-87799902,"num_objects":-189,"num_object_clones":-79,"num_object_copies":-378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-189,"num_whiteouts":-62,"num_read":1034,"num_read_kb":890,"num_write":176,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.073363"},"pg_stats":[{"pgid":"3.1f","version":"91'338","reported_seq":3956,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.267644+0000","last_change":"2024-09-13T07:13:54.267819+0000","last_active":"2024-09-13T07:13:54.267644+0000","last_peered":"2024-09-13T07:13:54.267644+0000","last_clean":"2024-09-13T07:13:54.267644+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:13:54.267644+0000","last_undegraded":"2024-09-13T07:13:54.267644+0000","last_fullsized":"2024-09-13T07:13:54.267644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010301820999999999,"stat_sum":{"num_bytes":10865935,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":20,"num_read":3980,"num_read_kb":17486,"num_write":349,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.1e","version":"91'344","reported_seq":4292,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.198934+0000","last_change":"2024-09-13T07:13:54.199242+0000","last_active":"2024-09-13T07:13:54.198934+0000","last_peered":"2024-09-13T07:13:54.198934+0000","last_clean":"2024-09-13T07:13:54.198934+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:13:54.198934+0000","last_undegraded":"2024-09-13T07:13:54.198934+0000","last_fullsized":"2024-09-13T07:13:54.198934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098182200000000003,"stat_sum":{"num_bytes":6740864,"num_objects":41,"num_object_clones":3,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":32,"num_read":4595,"num_read_kb":20236,"num_write":284,"num_write_kb":5817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.1d","version":"91'609","reported_seq":2545,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.201565+0000","last_change":"2024-09-13T07:13:54.201744+0000","last_active":"2024-09-13T07:13:54.201565+0000","last_peered":"2024-09-13T07:13:54.201565+0000","last_clean":"2024-09-13T07:13:54.201565+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:13:54.201565+0000","last_undegraded":"2024-09-13T07:13:54.201565+0000","last_fullsized":"2024-09-13T07:13:54.201565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074587600000000003,"stat_sum":{"num_bytes":14453468,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":20,"num_read":4244,"num_read_kb":14933,"num_write":848,"num_write_kb":8719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.1c","version":"92'486","reported_seq":3498,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.301098+0000","last_change":"2024-09-13T07:13:54.206167+0000","last_active":"2024-09-13T07:13:54.301098+0000","last_peered":"2024-09-13T07:13:54.301098+0000","last_clean":"2024-09-13T07:13:54.301098+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:13:54.301098+0000","last_undegraded":"2024-09-13T07:13:54.301098+0000","last_fullsized":"2024-09-13T07:13:54.301098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051971689999999997,"stat_sum":{"num_bytes":13124056,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":17,"num_read":3958,"num_read_kb":23264,"num_write":605,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.1b","version":"92'1058","reported_seq":4508,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.270103+0000","last_change":"2024-09-13T07:13:54.270299+0000","last_active":"2024-09-13T07:13:54.270103+0000","last_peered":"2024-09-13T07:13:54.270103+0000","last_clean":"2024-09-13T07:13:54.270103+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:13:54.270103+0000","last_undegraded":"2024-09-13T07:13:54.270103+0000","last_fullsized":"2024-09-13T07:13:54.270103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012916927999999999,"stat_sum":{"num_bytes":25395417,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":7908,"num_read_kb":35489,"num_write":1495,"num_write_kb":7824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.1a","version":"92'347","reported_seq":4228,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.340201+0000","last_change":"2024-09-13T07:13:54.204947+0000","last_active":"2024-09-13T07:13:54.340201+0000","last_peered":"2024-09-13T07:13:54.340201+0000","last_clean":"2024-09-13T07:13:54.340201+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:13:54.340201+0000","last_undegraded":"2024-09-13T07:13:54.340201+0000","last_fullsized":"2024-09-13T07:13:54.340201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011803110000000001,"stat_sum":{"num_bytes":8814627,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":4404,"num_read_kb":22951,"num_write":309,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.19","version":"90'328","reported_seq":5191,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.197599+0000","last_change":"2024-09-13T07:13:54.197918+0000","last_active":"2024-09-13T07:13:54.197599+0000","last_peered":"2024-09-13T07:13:54.197599+0000","last_clean":"2024-09-13T07:13:54.197599+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:13:54.197599+0000","last_undegraded":"2024-09-13T07:13:54.197599+0000","last_fullsized":"2024-09-13T07:13:54.197599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057947900000000004,"stat_sum":{"num_bytes":5643030,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":5010,"num_read_kb":22903,"num_write":348,"num_write_kb":19603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.18","version":"91'531","reported_seq":5372,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.200317+0000","last_change":"2024-09-13T07:13:54.200613+0000","last_active":"2024-09-13T07:13:54.200317+0000","last_peered":"2024-09-13T07:13:54.200317+0000","last_clean":"2024-09-13T07:13:54.200317+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:13:54.200317+0000","last_undegraded":"2024-09-13T07:13:54.200317+0000","last_fullsized":"2024-09-13T07:13:54.200317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001814041,"stat_sum":{"num_bytes":1706630,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":25,"num_read":7455,"num_read_kb":28863,"num_write":556,"num_write_kb":8200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.17","version":"92'486","reported_seq":3023,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.267607+0000","last_change":"2024-09-13T07:13:54.267607+0000","last_active":"2024-09-13T07:13:54.267607+0000","last_peered":"2024-09-13T07:13:54.267607+0000","last_clean":"2024-09-13T07:13:54.267607+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:13:54.267607+0000","last_undegraded":"2024-09-13T07:13:54.267607+0000","last_fullsized":"2024-09-13T07:13:54.267607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01013561,"stat_sum":{"num_bytes":7510051,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":4290,"num_read_kb":16557,"num_write":463,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.16","version":"91'233","reported_seq":1266,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.201081+0000","last_change":"2024-09-13T07:13:54.201272+0000","last_active":"2024-09-13T07:13:54.201081+0000","last_peered":"2024-09-13T07:13:54.201081+0000","last_clean":"2024-09-13T07:13:54.201081+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:13:54.201081+0000","last_undegraded":"2024-09-13T07:13:54.201081+0000","last_fullsized":"2024-09-13T07:13:54.201081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033354499999999998,"stat_sum":{"num_bytes":14437857,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":24,"num_read":843,"num_read_kb":7353,"num_write":213,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.15","version":"89'321","reported_seq":4552,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T07:13:52.179979+0000","last_change":"2024-09-13T07:13:52.180425+0000","last_active":"2024-09-13T07:13:52.179979+0000","last_peered":"2024-09-13T07:13:52.179979+0000","last_clean":"2024-09-13T07:13:52.179979+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:13:52.179979+0000","last_undegraded":"2024-09-13T07:13:52.179979+0000","last_fullsized":"2024-09-13T07:13:52.179979+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630639,"stat_sum":{"num_bytes":7530069,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":18,"num_read":4251,"num_read_kb":14694,"num_write":394,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"b"}]},{"pgid":"3.14","version":"91'392","reported_seq":2014,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.204257+0000","last_change":"2024-09-13T07:13:54.204428+0000","last_active":"2024-09-13T07:13:54.204257+0000","last_peered":"2024-09-13T07:13:54.204257+0000","last_clean":"2024-09-13T07:13:54.204257+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:13:54.204257+0000","last_undegraded":"2024-09-13T07:13:54.204257+0000","last_fullsized":"2024-09-13T07:13:54.204257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061556999999999996,"stat_sum":{"num_bytes":22951853,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":2555,"num_read_kb":13816,"num_write":427,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"1.7","version":"0'0","reported_seq":90,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T07:13:53.153824+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:13:53.153824+0000","last_peered":"2024-09-13T07:13:53.153824+0000","last_clean":"2024-09-13T07:13:53.153824+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:13:53.153824+0000","last_undegraded":"2024-09-13T07:13:53.153824+0000","last_fullsized":"2024-09-13T07:13:53.153824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"91'393","reported_seq":3601,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.201789+0000","last_change":"2024-09-13T07:13:54.202009+0000","last_active":"2024-09-13T07:13:54.201789+0000","last_peered":"2024-09-13T07:13:54.201789+0000","last_clean":"2024-09-13T07:13:54.201789+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:13:54.201789+0000","last_undegraded":"2024-09-13T07:13:54.201789+0000","last_fullsized":"2024-09-13T07:13:54.201789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007263,"stat_sum":{"num_bytes":6890628,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":17,"num_read":4677,"num_read_kb":16003,"num_write":442,"num_write_kb":4945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T07:13:53.153944+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:13:53.153944+0000","last_peered":"2024-09-13T07:13:53.153944+0000","last_clean":"2024-09-13T07:13:53.153944+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:13:53.153944+0000","last_undegraded":"2024-09-13T07:13:53.153944+0000","last_fullsized":"2024-09-13T07:13:53.153944+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"92'523","reported_seq":4248,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.197992+0000","last_change":"2024-09-13T07:13:54.197992+0000","last_active":"2024-09-13T07:13:54.197992+0000","last_peered":"2024-09-13T07:13:54.197992+0000","last_clean":"2024-09-13T07:13:54.197992+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:13:54.197992+0000","last_undegraded":"2024-09-13T07:13:54.197992+0000","last_fullsized":"2024-09-13T07:13:54.197992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065696000000000005,"stat_sum":{"num_bytes":4169314,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":5255,"num_read_kb":14124,"num_write":635,"num_write_kb":8122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T07:13:53.146812+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:13:53.146812+0000","last_peered":"2024-09-13T07:13:53.146812+0000","last_clean":"2024-09-13T07:13:53.146812+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:13:53.146812+0000","last_undegraded":"2024-09-13T07:13:53.146812+0000","last_fullsized":"2024-09-13T07:13:53.146812+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"90'367","reported_seq":884,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.199352+0000","last_change":"2024-09-13T07:13:54.199804+0000","last_active":"2024-09-13T07:13:54.199352+0000","last_peered":"2024-09-13T07:13:54.199352+0000","last_clean":"2024-09-13T07:13:54.199352+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:13:54.199352+0000","last_undegraded":"2024-09-13T07:13:54.199352+0000","last_fullsized":"2024-09-13T07:13:54.199352+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015516219999999999,"stat_sum":{"num_bytes":4428430,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":21,"num_read":1167,"num_read_kb":5375,"num_write":463,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T07:13:53.143691+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:13:53.143691+0000","last_peered":"2024-09-13T07:13:53.143691+0000","last_clean":"2024-09-13T07:13:53.143691+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:13:53.143691+0000","last_undegraded":"2024-09-13T07:13:53.143691+0000","last_fullsized":"2024-09-13T07:13:53.143691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"92'352","reported_seq":7717,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.323077+0000","last_change":"2024-09-13T07:13:54.265741+0000","last_active":"2024-09-13T07:13:54.323077+0000","last_peered":"2024-09-13T07:13:54.323077+0000","last_clean":"2024-09-13T07:13:54.323077+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:13:54.323077+0000","last_undegraded":"2024-09-13T07:13:54.323077+0000","last_fullsized":"2024-09-13T07:13:54.323077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0083001559999999995,"stat_sum":{"num_bytes":2271378,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":13,"num_read":7463,"num_read_kb":20704,"num_write":357,"num_write_kb":18762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.1","version":"92'561","reported_seq":4579,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.262415+0000","last_change":"2024-09-13T07:13:54.262415+0000","last_active":"2024-09-13T07:13:54.262415+0000","last_peered":"2024-09-13T07:13:54.262415+0000","last_clean":"2024-09-13T07:13:54.262415+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:13:54.262415+0000","last_undegraded":"2024-09-13T07:13:54.262415+0000","last_fullsized":"2024-09-13T07:13:54.262415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0051612660000000003,"stat_sum":{"num_bytes":6704872,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":5434,"num_read_kb":29597,"num_write":798,"num_write_kb":11467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"2.0","version":"17'117","reported_seq":207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T07:13:51.192793+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:13:51.192793+0000","last_peered":"2024-09-13T07:13:51.192793+0000","last_clean":"2024-09-13T07:13:51.192793+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:13:51.192793+0000","last_undegraded":"2024-09-13T07:13:51.192793+0000","last_fullsized":"2024-09-13T07:13:51.192793+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T07:13:53.142240+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:13:53.142240+0000","last_peered":"2024-09-13T07:13:53.142240+0000","last_clean":"2024-09-13T07:13:53.142240+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:13:53.142240+0000","last_undegraded":"2024-09-13T07:13:53.142240+0000","last_fullsized":"2024-09-13T07:13:53.142240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"92'439","reported_seq":2574,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.200637+0000","last_change":"2024-09-13T07:13:54.201127+0000","last_active":"2024-09-13T07:13:54.200637+0000","last_peered":"2024-09-13T07:13:54.200637+0000","last_clean":"2024-09-13T07:13:54.200637+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:13:54.200637+0000","last_undegraded":"2024-09-13T07:13:54.200637+0000","last_fullsized":"2024-09-13T07:13:54.200637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037301159999999999,"stat_sum":{"num_bytes":19104286,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":3079,"num_read_kb":26596,"num_write":613,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"1.2","version":"17'2","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T07:13:51.192967+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:13:51.192967+0000","last_peered":"2024-09-13T07:13:51.192967+0000","last_clean":"2024-09-13T07:13:51.192967+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:13:51.192967+0000","last_undegraded":"2024-09-13T07:13:51.192967+0000","last_fullsized":"2024-09-13T07:13:51.192967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'533","reported_seq":4189,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.309063+0000","last_change":"2024-09-13T07:13:54.204765+0000","last_active":"2024-09-13T07:13:54.309063+0000","last_peered":"2024-09-13T07:13:54.309063+0000","last_clean":"2024-09-13T07:13:54.309063+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:13:54.309063+0000","last_undegraded":"2024-09-13T07:13:54.309063+0000","last_fullsized":"2024-09-13T07:13:54.309063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061638899999999996,"stat_sum":{"num_bytes":1982501,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":23,"num_read":4706,"num_read_kb":23480,"num_write":615,"num_write_kb":25190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T07:13:53.143126+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:13:53.143126+0000","last_peered":"2024-09-13T07:13:53.143126+0000","last_clean":"2024-09-13T07:13:53.143126+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:13:53.143126+0000","last_undegraded":"2024-09-13T07:13:53.143126+0000","last_fullsized":"2024-09-13T07:13:53.143126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'448","reported_seq":5064,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.203682+0000","last_change":"2024-09-13T07:13:54.203682+0000","last_active":"2024-09-13T07:13:54.203682+0000","last_peered":"2024-09-13T07:13:54.203682+0000","last_clean":"2024-09-13T07:13:54.203682+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:13:54.203682+0000","last_undegraded":"2024-09-13T07:13:54.203682+0000","last_fullsized":"2024-09-13T07:13:54.203682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.005445915,"stat_sum":{"num_bytes":28950515,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":5731,"num_read_kb":26704,"num_write":630,"num_write_kb":28888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T07:13:51.192686+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:13:51.192686+0000","last_peered":"2024-09-13T07:13:51.192686+0000","last_clean":"2024-09-13T07:13:51.192686+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:13:51.192686+0000","last_undegraded":"2024-09-13T07:13:51.192686+0000","last_fullsized":"2024-09-13T07:13:51.192686+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"91'443","reported_seq":2165,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.257473+0000","last_change":"2024-09-13T07:13:54.257832+0000","last_active":"2024-09-13T07:13:54.257473+0000","last_peered":"2024-09-13T07:13:54.257473+0000","last_clean":"2024-09-13T07:13:54.257473+0000","last_became_active":"2024-09-13T07:13:28.445371+0000","last_became_peered":"2024-09-13T07:13:28.445371+0000","last_unstale":"2024-09-13T07:13:54.257473+0000","last_undegraded":"2024-09-13T07:13:54.257473+0000","last_fullsized":"2024-09-13T07:13:54.257473+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:22.087974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064599999999999998,"stat_sum":{"num_bytes":5804491,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":2673,"num_read_kb":13164,"num_write":510,"num_write_kb":9269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.9","version":"86'500","reported_seq":1410,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.203943+0000","last_change":"2024-09-13T07:13:54.204225+0000","last_active":"2024-09-13T07:13:54.203943+0000","last_peered":"2024-09-13T07:13:54.203943+0000","last_clean":"2024-09-13T07:13:54.203943+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:13:54.203943+0000","last_undegraded":"2024-09-13T07:13:54.203943+0000","last_fullsized":"2024-09-13T07:13:54.203943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450486,"stat_sum":{"num_bytes":3039202,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":2264,"num_read_kb":8187,"num_write":748,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.a","version":"92'776","reported_seq":3465,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.330788+0000","last_change":"2024-09-13T07:13:54.330788+0000","last_active":"2024-09-13T07:13:54.330788+0000","last_peered":"2024-09-13T07:13:54.330788+0000","last_clean":"2024-09-13T07:13:54.330788+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:13:54.330788+0000","last_undegraded":"2024-09-13T07:13:54.330788+0000","last_fullsized":"2024-09-13T07:13:54.330788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010045950000000001,"stat_sum":{"num_bytes":8284861,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":6485,"num_read_kb":19518,"num_write":956,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.b","version":"91'820","reported_seq":3705,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.199899+0000","last_change":"2024-09-13T07:13:54.200220+0000","last_active":"2024-09-13T07:13:54.199899+0000","last_peered":"2024-09-13T07:13:54.199899+0000","last_clean":"2024-09-13T07:13:54.199899+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:13:54.199899+0000","last_undegraded":"2024-09-13T07:13:54.199899+0000","last_fullsized":"2024-09-13T07:13:54.199899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411914,"stat_sum":{"num_bytes":26411463,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":7411,"num_read_kb":20077,"num_write":964,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.c","version":"91'307","reported_seq":2299,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.201311+0000","last_change":"2024-09-13T07:13:54.201522+0000","last_active":"2024-09-13T07:13:54.201311+0000","last_peered":"2024-09-13T07:13:54.201311+0000","last_clean":"2024-09-13T07:13:54.201311+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:13:54.201311+0000","last_undegraded":"2024-09-13T07:13:54.201311+0000","last_fullsized":"2024-09-13T07:13:54.201311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054445100000000005,"stat_sum":{"num_bytes":20348199,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":2382,"num_read_kb":20225,"num_write":374,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.d","version":"88'344","reported_seq":2251,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T07:13:52.183875+0000","last_change":"2024-09-13T07:13:52.184155+0000","last_active":"2024-09-13T07:13:52.183875+0000","last_peered":"2024-09-13T07:13:52.183875+0000","last_clean":"2024-09-13T07:13:52.183875+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:13:52.183875+0000","last_undegraded":"2024-09-13T07:13:52.183875+0000","last_fullsized":"2024-09-13T07:13:52.183875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041782620000000003,"stat_sum":{"num_bytes":7299514,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":2306,"num_read_kb":8374,"num_write":325,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"b"}]},{"pgid":"3.e","version":"90'650","reported_seq":1877,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T07:13:52.183814+0000","last_change":"2024-09-13T07:13:52.183814+0000","last_active":"2024-09-13T07:13:52.183814+0000","last_peered":"2024-09-13T07:13:52.183814+0000","last_clean":"2024-09-13T07:13:52.183814+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:13:52.183814+0000","last_undegraded":"2024-09-13T07:13:52.183814+0000","last_fullsized":"2024-09-13T07:13:52.183814+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0039045099999999999,"stat_sum":{"num_bytes":5433628,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":5953,"num_read_kb":25404,"num_write":786,"num_write_kb":9159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"b"}]},{"pgid":"3.f","version":"89'1025","reported_seq":7318,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T07:13:52.184235+0000","last_change":"2024-09-13T07:13:52.184486+0000","last_active":"2024-09-13T07:13:52.184235+0000","last_peered":"2024-09-13T07:13:52.184235+0000","last_clean":"2024-09-13T07:13:52.184235+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:13:52.184235+0000","last_undegraded":"2024-09-13T07:13:52.184235+0000","last_fullsized":"2024-09-13T07:13:52.184235+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044385400000000004,"stat_sum":{"num_bytes":12748390,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":10062,"num_read_kb":17026,"num_write":1660,"num_write_kb":24994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"b"}]},{"pgid":"3.10","version":"88'438","reported_seq":2420,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.329944+0000","last_change":"2024-09-13T07:13:54.330181+0000","last_active":"2024-09-13T07:13:54.329944+0000","last_peered":"2024-09-13T07:13:54.329944+0000","last_clean":"2024-09-13T07:13:54.329944+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:13:54.329944+0000","last_undegraded":"2024-09-13T07:13:54.329944+0000","last_fullsized":"2024-09-13T07:13:54.329944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037943500000000002,"stat_sum":{"num_bytes":11946854,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":19,"num_read":3233,"num_read_kb":21310,"num_write":482,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.11","version":"85'442","reported_seq":3778,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T07:13:52.179848+0000","last_change":"2024-09-13T07:13:52.180309+0000","last_active":"2024-09-13T07:13:52.179848+0000","last_peered":"2024-09-13T07:13:52.179848+0000","last_clean":"2024-09-13T07:13:52.179848+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:13:52.179848+0000","last_undegraded":"2024-09-13T07:13:52.179848+0000","last_fullsized":"2024-09-13T07:13:52.179848+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057728800000000002,"stat_sum":{"num_bytes":14192789,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":4025,"num_read_kb":31952,"num_write":550,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"b"}]},{"pgid":"3.12","version":"92'278","reported_seq":1784,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T07:13:54.262662+0000","last_change":"2024-09-13T07:13:54.263124+0000","last_active":"2024-09-13T07:13:54.262662+0000","last_peered":"2024-09-13T07:13:54.262662+0000","last_clean":"2024-09-13T07:13:54.262662+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:13:54.262662+0000","last_undegraded":"2024-09-13T07:13:54.262662+0000","last_fullsized":"2024-09-13T07:13:54.262662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0058458199999999998,"stat_sum":{"num_bytes":10931841,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":22,"num_read":1303,"num_read_kb":13724,"num_write":234,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.13","version":"90'618","reported_seq":3853,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T07:13:52.360417+0000","last_change":"2024-09-13T07:13:52.183904+0000","last_active":"2024-09-13T07:13:52.360417+0000","last_peered":"2024-09-13T07:13:52.360417+0000","last_clean":"2024-09-13T07:13:52.360417+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:13:52.360417+0000","last_undegraded":"2024-09-13T07:13:52.360417+0000","last_fullsized":"2024-09-13T07:13:52.360417+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.004074378,"stat_sum":{"num_bytes":17578334,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":14,"num_read":5386,"num_read_kb":24522,"num_write":737,"num_write_kb":12627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":357695347,"num_objects":889,"num_object_clones":96,"num_object_copies":1778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":889,"num_whiteouts":554,"num_read":144488,"num_read_kb":624611,"num_write":19170,"num_write_kb":356386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11603968,"data_stored":62258607,"data_compressed":2452526,"data_compressed_allocated":4911104,"data_compressed_original":56794290,"omap_allocated":7126,"internal_metadata":0},"log_size":15730,"ondisk_log_size":15730,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266697,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40500,"kb_used_data":3584,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94331340,"statfs":{"total":96636764160,"available":96595292160,"internally_reserved":0,"allocated":3670016,"data_stored":20808978,"data_compressed":609186,"data_compressed_allocated":1515520,"data_compressed_original":19394560,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":1490944,"data_stored":2089253,"data_compressed":54833,"data_compressed_allocated":151552,"data_compressed_original":1372160,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.376}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.29899999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.309},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.28199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46080,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94325760,"statfs":{"total":96636764160,"available":96589578240,"internally_reserved":0,"allocated":2736128,"data_stored":9297263,"data_compressed":313613,"data_compressed_allocated":614400,"data_compressed_original":7949913,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.626},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.33000000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78768,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293072,"statfs":{"total":96636764160,"available":96556105728,"internally_reserved":0,"allocated":3383296,"data_stored":10420931,"data_compressed":422629,"data_compressed_allocated":819200,"data_compressed_original":8679424,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.433},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.497}]},{"osd":5,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:13:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29656,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342184,"statfs":{"total":96636764160,"available":96606396416,"internally_reserved":0,"allocated":2695168,"data_stored":11082097,"data_compressed":436573,"data_compressed_allocated":749568,"data_compressed_original":9885609,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45600000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.749},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36668,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335172,"statfs":{"total":96636764160,"available":96599216128,"internally_reserved":0,"allocated":1490944,"data_stored":1587855,"data_compressed":38446,"data_compressed_allocated":86016,"data_compressed_original":835584,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.26700000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:13:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41380,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330460,"statfs":{"total":96636764160,"available":96594391040,"internally_reserved":0,"allocated":2117632,"data_stored":5438590,"data_compressed":380215,"data_compressed_allocated":528384,"data_compressed_original":4534272,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:13:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52868,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318972,"statfs":{"total":96636764160,"available":96582627328,"internally_reserved":0,"allocated":3735552,"data_stored":16675908,"data_compressed":862150,"data_compressed_allocated":1331200,"data_compressed_original":15046320,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.3090000000000002,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":4.54,"5min":4.54,"15min":4.54},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Sep 13 07:13:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2297856,"data_stored":12693907,"data_compressed":589118,"data_compressed_allocated":1048576,"data_compressed_original":11638448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":839680,"data_stored":3222031,"data_compressed":224115,"data_compressed_allocated":368640,"data_compressed_original":2863104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":1079589,"data_compressed":38446,"data_compressed_allocated":86016,"data_compressed_original":835584,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":10567575,"data_compressed":436573,"data_compressed_allocated":749568,"data_compressed_original":9885609,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2293760,"data_stored":9906409,"data_compressed":422629,"data_compressed_allocated":819200,"data_compressed_original":8679424,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1523712,"data_stored":7691357,"data_compressed":248011,"data_compressed_allocated":540672,"data_compressed_original":6901337,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":1580987,"data_compressed":54833,"data_compressed_allocated":151552,"data_compressed_original":1372160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2215936,"data_stored":15516752,"data_compressed":438801,"data_compressed_allocated":1146880,"data_compressed_original":14618624,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:13:57.647 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:57.647 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:57.650 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:57.652+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:57.656 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:13:57.752 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:57.752+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:57.758 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:57.758 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:57.767 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:13:57.853 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:57.853+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:57.910 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:57.910 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:57.919 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:13:57.962 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:57.962+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.021 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.030 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:13:58.071 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:58.072+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:58.155 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:13:58.155+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:13:58.179 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.179 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.188 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:13:58.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:13:58.271 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:58.271+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:58.293 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.294 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:13:58.357 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:58.356+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:58.404 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.405 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.413 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:13:58.458 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:58.458+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.556 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.558 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:58.558+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:58.564 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:13:58.659 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:58.660+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:58.667 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.667 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:13:58.760 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:58.760+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:58.784 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.784 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:13:58.861 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:13:58.862+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:13:58.900 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:58.900 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:58.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:13:58.962 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:13:58.962+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:13:59.016 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:59.016 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:59.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:13:59.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:13:59.065 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:59.065+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:59.164 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:13:59.165+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:13:59.173 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:13:59.173 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:13:59.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:13:59.188 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:13:59.189 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:13:59.206 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:13:59.206 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:13:59.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:13:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:13:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:13:59.245 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:13:59.245 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:13:59.262 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:13:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:13:59.265 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:59.266+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:59.284 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:13:59.284 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:13:59.303 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:59.303 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:59.304 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:13:59.304 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:13:59.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:13:59.323 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:13:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:13:59.366 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:59.367+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:59.434 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49523 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2699264~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.992048052000001, 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056285390999999997, 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49523 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442061+0000", 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442063+0000", 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9379999999999999e-06 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442068+0000", 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.442073+0000", 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:13:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497684+0000", 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055610930000000003 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.497759+0000", 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5199999999999998e-05 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.498346+0000", 2024-09-13T07:13:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058719100000000004 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49609 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 4075520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.973983721000003, 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055911012000000003, 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49609 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460125+0000", 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460127+0000", 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1430000000000001e-06 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460133+0000", 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5729999999999998e-06 2024-09-13T07:13:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.460138+0000", 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515783+0000", 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055645873999999998 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.515861+0000", 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7979000000000007e-05 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.516036+0000", 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017504900000000001 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49618 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 999424~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.971469313, 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055816179, 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49618 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462640+0000", 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462639+0000", 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462642+0000", 2024-09-13T07:13:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462647+0000", 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462652+0000", 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518291+0000", 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055639051000000002 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518368+0000", 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6913e-05 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518456+0000", 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8219000000000004e-05 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49619 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.971452888000002, 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055989537999999998, 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49619 2024-09-13T07:13:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462656+0000", 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462659+0000", 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462665+0000", 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462669+0000", 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518483+0000", 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055813437 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518559+0000", 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5981000000000003e-05 2024-09-13T07:13:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518646+0000", 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7175999999999993e-05 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49620 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1048576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.971434983000002, 2024-09-13T07:13:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056152684000000001, 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49620 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462674+0000", 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462676+0000", 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462683+0000", 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9269999999999999e-06 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462688+0000", 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518672+0000", 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055983988999999998 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518747+0000", 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4633000000000002e-05 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518827+0000", 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000002e-05 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49621 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1073152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.971416235, 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056323023999999999, 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49621 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462693+0000", 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462695+0000", 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462701+0000", 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462706+0000", 2024-09-13T07:13:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6530000000000002e-06 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518854+0000", 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056147858000000002 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.518934+0000", 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0813e-05 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519016+0000", 2024-09-13T07:13:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1434999999999995e-05 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49622 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1097728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.971398948999997, 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056490088000000001, 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49622 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462710+0000", 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462712+0000", 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462718+0000", 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.462722+0000", 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519042+0000", 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056320041000000001 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519120+0000", 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7607000000000004e-05 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.519200+0000", 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0166999999999993e-05 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49640 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1318912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.967594245999997, 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056480083, 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49640 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466515+0000", 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466514+0000", 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466517+0000", 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7070000000000001e-06 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466524+0000", 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466529+0000", 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520731+0000", 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054202469000000003 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.520808+0000", 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6854000000000007e-05 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522995+0000", 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00218681 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49653 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1490944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.967473034000001, 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055956026999999998, 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49653 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466636+0000", 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466638+0000", 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466643+0000", 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.466648+0000", 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522424+0000", 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055775681000000001 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522499+0000", 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5828000000000007e-05 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.522592+0000", 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2781000000000005e-05 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49674 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1662976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.966059645000001, 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056005177000000003, 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49674 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468049+0000", 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468052+0000", 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.497e-06 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468058+0000", 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2430000000000002e-06 2024-09-13T07:13:59.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.468063+0000", 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523872+0000", 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055808706999999999 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.523960+0000", 2024-09-13T07:13:59.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8919999999999996e-05 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.524055+0000", 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4222999999999996e-05 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49695 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.963773983999999, 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056046466000000003, 2024-09-13T07:13:59.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49695 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:59.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470335+0000", 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470338+0000", 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470345+0000", 2024-09-13T07:13:59.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4100000000000002e-06 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470351+0000", 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525756+0000", 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055404954999999999 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.525819+0000", 2024-09-13T07:13:59.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2779000000000003e-05 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526382+0000", 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056291800000000003 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49696 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.963753285999999, 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056222355000000002, 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49696 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470356+0000", 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470355+0000", 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470358+0000", 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470364+0000", 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:13:59.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.470369+0000", 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526411+0000", 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056041476999999999 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526489+0000", 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7787000000000006e-05 2024-09-13T07:13:59.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.526578+0000", 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9606999999999998e-05 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49704 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2056192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.962127434999999, 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055974318000000002, 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49704 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471982+0000", 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471981+0000", 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471984+0000", 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471990+0000", 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7229999999999998e-06 2024-09-13T07:13:59.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471995+0000", 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527788+0000", 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055793091000000003 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527861+0000", 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3038000000000001e-05 2024-09-13T07:13:59.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527956+0000", 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4536000000000004e-05 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49706 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2080768~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.962109628, 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056135461999999997, 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49706 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.471999+0000", 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472006+0000", 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472012+0000", 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472017+0000", 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.527980+0000", 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055963185999999998 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528050+0000", 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979000000000002e-05 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528135+0000", 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4594999999999999e-05 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49708 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2105344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.962087467000003, 2024-09-13T07:13:59.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056284837999999997, 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49708 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472022+0000", 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472021+0000", 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472024+0000", 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.261e-06 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472029+0000", 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472034+0000", 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0069999999999997e-06 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528161+0000", 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056126285999999997 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528232+0000", 2024-09-13T07:13:59.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1000000000000005e-05 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528306+0000", 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4876000000000006e-05 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49711 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2129920~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.962069952999997, 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444877999999997, 2024-09-13T07:13:59.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49711 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472039+0000", 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472041+0000", 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472047+0000", 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472052+0000", 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528331+0000", 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056278504 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528399+0000", 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7941000000000006e-05 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528484+0000", 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5249999999999999e-05 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49713 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2154496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.961898640000001, 2024-09-13T07:13:59.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056444079000000001, 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49713 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472210+0000", 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472214+0000", 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:13:59.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472228+0000", 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3964999999999999e-05 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.472236+0000", 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4109999999999999e-06 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528509+0000", 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056273532000000001 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528581+0000", 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1338e-05 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.528655+0000", 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3858999999999999e-05 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49728 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2326528~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.960024738999998, 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055971704999999997, 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49728 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474084+0000", 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474087+0000", 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474093+0000", 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.474098+0000", 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529880+0000", 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055782102 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.529967+0000", 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6853000000000003e-05 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.530056+0000", 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9430000000000003e-05 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:49767 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 2695168~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.955935062999998, 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062028049000000002, 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49767 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:59.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478174+0000", 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478176+0000", 2024-09-13T07:13:59.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5670000000000002e-06 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478182+0000", 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.478187+0000", 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532704+0000", 2024-09-13T07:13:59.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054517200000000002 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.532779+0000", 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5244e-05 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.540202+0000", 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074227429999999999 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:50078 3.4 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.942132882999999, 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055871918999999999, 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50078 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491976+0000", 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491979+0000", 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491986+0000", 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.491991+0000", 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544160+0000", 2024-09-13T07:13:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052168953999999997 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.544241+0000", 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1213999999999999e-05 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.547848+0000", 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003606663 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.519 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:59.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:13:59.520 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:13:59.520+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:13:59.621 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:13:59.622+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:13:59.667 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:59.667 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:59.667 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:13:59.667 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:59.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:13:59.721 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:13:59.721+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:13:59.783 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:59.783 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:13:59.783 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:13:59.783 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:13:59.783 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:13:59.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:13:59.822 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:59.823+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:59.923 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:13:59.924+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.9 epoch 25/19 rep_tid 26 entries 25'7 (0'0) error 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head by client.4467.0:1092 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 25'5)", 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.642344270999999, 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035999920000000002, 2024-09-13T07:13:59.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297922+0000", 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297921+0000", 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297930+0000", 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3500000000000003e-06 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297950+0000", 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9667e-05 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.297963+0000", 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2981e-05 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.298026+0000", 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2788999999999998e-05 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.301522+0000", 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034961039999999999 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1097 3.0 e25/19 3:0092f09a:::rbd_data.1173f77ecad8.0000000000000052:head v 25'6, mlcod=24'4)", 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.640866246000002, 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048975640000000001, 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299400+0000", 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299399+0000", 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299406+0000", 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3479999999999998e-06 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299420+0000", 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3881e-05 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299427+0000", 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.323e-06 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299469+0000", 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2178999999999997e-05 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.299492+0000", 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3010000000000002e-05 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304257+0000", 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047646579999999997 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.304297+0000", 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0216000000000003e-05 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:1103 3.9 e25/19 3:95d1f773:::rbd_data.1173f77ecad8.0000000000000054:head v 25'8, mlcod=25'6)", 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.638049197000001, 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048558100000000003, 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302217+0000", 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302216+0000", 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302223+0000", 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7510000000000008e-06 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302237+0000", 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3986e-05 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302257+0000", 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9321e-05 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302305+0000", 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8241000000000001e-05 2024-09-13T07:13:59.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.302324+0000", 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8746e-05 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307033+0000", 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004709609 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:37.307073+0000", 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9468000000000002e-05 2024-09-13T07:13:59.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:3064 3.4 e27/19 3:27ec9434:::rbd_object_map.11734c6039c3:head v 27'106, mlcod=27'104)", 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.068620237000005, 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003328896, 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871646+0000", 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871645+0000", 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871652+0000", 2024-09-13T07:13:59.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.754e-06 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871664+0000", 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2476e-05 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871670+0000", 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2269999999999998e-06 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871707+0000", 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6260000000000002e-05 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.871724+0000", 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7125e-05 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874935+0000", 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032106410000000002 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:40.874975+0000", 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0179000000000003e-05 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4077 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'102, mlcod=29'100)", 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.172796009999999, 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033784280000000002, 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767470+0000", 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767469+0000", 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767481+0000", 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1413000000000001e-05 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767492+0000", 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080000000000001e-05 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767498+0000", 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767537+0000", 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9295000000000002e-05 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.767560+0000", 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2776999999999999e-05 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770806+0000", 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032457710000000002 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.770848+0000", 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2765999999999997e-05 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4163 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'114, mlcod=29'110)", 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.125448716999998, 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047390619999999996, 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:59.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814817+0000", 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814820+0000", 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5729999999999999e-06 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814828+0000", 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4699999999999996e-06 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814831+0000", 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814861+0000", 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9731e-05 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.814872+0000", 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819531+0000", 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046589780000000003 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.819556+0000", 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5717000000000001e-05 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:4196 3.9 e29/19 3:97afc258:::rbd_object_map.11736f46b2f8:head v 29'123, mlcod=29'119)", 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.091950187999998, 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049391649999999997, 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848316+0000", 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848315+0000", 2024-09-13T07:13:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848319+0000", 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848327+0000", 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848330+0000", 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4529999999999998e-06 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848370+0000", 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9786999999999997e-05 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.848390+0000", 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211999999999999e-05 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853231+0000", 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048401449999999997 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:42.853255+0000", 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4408000000000001e-05 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:7729 3.0 e34/19 3:02f6b5f6:::rbd_object_map.117344605995:head v 34'60, mlcod=34'54)", 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.504034046000001, 2024-09-13T07:13:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033788030000000001, 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436232+0000", 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436231+0000", 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436237+0000", 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436247+0000", 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0053e-05 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436254+0000", 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6579999999999996e-06 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436307+0000", 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3358e-05 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.436325+0000", 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7819e-05 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439574+0000", 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003248599 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:12:49.439611+0000", 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7055999999999999e-05 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:35168 3.4 e43/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:head v 43'284, mlcod=43'281)", 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.233472179000003, 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004516829, 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706794+0000", 2024-09-13T07:13:59.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706792+0000", 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706854+0000", 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1915999999999995e-05 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706890+0000", 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5818000000000002e-05 2024-09-13T07:13:59.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706901+0000", 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1433999999999999e-05 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706972+0000", 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1072e-05 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.706997+0000", 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4369999999999999e-05 2024-09-13T07:13:59.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711269+0000", 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042726120000000003 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:00.711311+0000", 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1523999999999997e-05 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:64537 3.9 e47/19 3:97a74db9:::rbd_data.117332306d66.0000000000000002:head v 47'372, mlcod=45'370)", 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.320858747000003, 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092884539999999998, 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.619407+0000", 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.624988+0000", 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055812279999999997 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625018+0000", 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9496999999999999e-05 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625031+0000", 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3390000000000001e-05 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625078+0000", 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7290000000000003e-05 2024-09-13T07:13:59.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.625096+0000", 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8091999999999999e-05 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628662+0000", 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035652449999999999 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:05.628696+0000", 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4035999999999997e-05 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:97494 3.0 e67/19 3:02ad2de2:::rbd_data.117387780896.0000000000000002:head v 67'375, mlcod=66'372)", 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.066657373999998, 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005628437, 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873609+0000", 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.873608+0000", 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875823+0000", 2024-09-13T07:13:59.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022143660000000002 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875843+0000", 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9864000000000002e-05 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875854+0000", 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1345999999999999e-05 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875918+0000", 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4151000000000003e-05 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.875939+0000", 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1016999999999999e-05 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879194+0000", 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032546820000000001 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:28.879237+0000", 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3353000000000003e-05 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1145 3.4 e76/19 3:27c86583:::rbd_data.11738a001609.0000000000000046:22 v 76'478, mlcod=71'472)", 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.424658714, 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063983709999999999, 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515607+0000", 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515618+0000", 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0787e-05 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515629+0000", 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0905e-05 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.515634+0000", 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1710000000000001e-06 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.516333+0000", 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00069910600000000001 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.516351+0000", 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7822999999999999e-05 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.521968+0000", 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056176519999999999 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:38.522006+0000", 2024-09-13T07:13:59.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7246000000000002e-05 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1172 3.0 e84/19 3:075a22d2:::rbd_data.1173d7ab5a36.000000000000000f:1a v 84'423, mlcod=80'417)", 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.153288+0000", 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.78697844, 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033929540000000001, 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153288+0000", 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153288+0000", 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153287+0000", 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153292+0000", 2024-09-13T07:13:59.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3800000000000002e-06 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153300+0000", 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.841e-06 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153307+0000", 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153372+0000", 2024-09-13T07:13:59.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5679999999999995e-05 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.153400+0000", 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7545000000000001e-05 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156642+0000", 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032426759999999999 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156681+0000", 2024-09-13T07:13:59.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8112999999999999e-05 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.4.0:2822 3.9 e84/19 3:91ae4f65:::rbd_data.1173d7ab5a36.0000000000000022:1a v 84'496, mlcod=82'492)", 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.78485762, 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044564410000000002, 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155408+0000", 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155413+0000", 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0359999999999997e-06 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155427+0000", 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3675e-05 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155435+0000", 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8739999999999995e-06 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155490+0000", 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155507+0000", 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7204999999999999e-05 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.159833+0000", 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043255990000000003 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.159865+0000", 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2325e-05 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1173 3.4 e84/19 3:2217e031:::rbd_data.1173d7ab5a36.0000000000000036:1a v 84'488, mlcod=82'484)", 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.155592+0000", 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.784674123, 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069439200000000001, 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:13:59.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155592+0000", 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155592+0000", 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155591+0000", 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:13:59.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155607+0000", 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5556000000000001e-05 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155618+0000", 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0842999999999999e-05 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155623+0000", 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0490000000000003e-06 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:13:59.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155663+0000", 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0085e-05 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155685+0000", 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2693000000000001e-05 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162499+0000", 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068136239999999999 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162536+0000", 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6789000000000003e-05 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.4.0:2824 3.9 e84/19 3:95c5389a:::rbd_data.1173d7ab5a36.000000000000000c:1a v 84'498, mlcod=82'492)", 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.155976+0000", 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.784290424, 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064142050000000001, 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155976+0000", 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155976+0000", 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155975+0000", 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155980+0000", 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155990+0000", 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2690000000000005e-06 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.155994+0000", 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5639999999999998e-06 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156029+0000", 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4696999999999997e-05 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156042+0000", 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869e-05 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162346+0000", 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063042840000000003 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.162390+0000", 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3649999999999997e-05 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1174 3.4 e84/19 3:25e9d115:::rbd_data.1173d7ab5a36.000000000000000d:1a v 84'490, mlcod=82'484)", 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:46.156226+0000", 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.784040349, 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089488970000000008, 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156226+0000", 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156226+0000", 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156225+0000", 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156231+0000", 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3199999999999996e-06 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156243+0000", 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1294000000000001e-05 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.156249+0000", 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6560000000000003e-06 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.157990+0000", 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017409669999999999 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.158010+0000", 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0086999999999999e-05 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.165137+0000", 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071266180000000004 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:46.165175+0000", 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7642999999999999e-05 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:1207 3.4 e89/19 3:207ace13:::rbd_data.1173fee7d7bf.0000000000000005:15 v 89'516, mlcod=86'513)", 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:51.180199+0000", 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.7600672559999992, 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033445800000000002, 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180199+0000", 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180199+0000", 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180198+0000", 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180207+0000", 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7010000000000002e-06 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180218+0000", 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.136e-05 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180226+0000", 2024-09-13T07:14:00.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2300000000000008e-06 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180287+0000", 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0825000000000001e-05 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.180308+0000", 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0727e-05 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.183495+0000", 2024-09-13T07:14:00.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031865639999999998 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:51.183543+0000", 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8798000000000003e-05 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:107741 3.9 e94/19 3:95c0f194:::rbd_data.1173f6773d36.000000000000000d:head v 94'501, mlcod=85'499)", 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6530659229999998, 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038223659999999998, 2024-09-13T07:14:00.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287200+0000", 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287204+0000", 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7520000000000004e-06 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287215+0000", 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287225+0000", 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3689999999999996e-06 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287265+0000", 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0309999999999999e-05 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.287278+0000", 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2874000000000001e-05 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290981+0000", 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037032480000000001 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.291022+0000", 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1519e-05 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4467.0:107747 3.0 e94/19 3:07d9a7ed:::rbd_data.1173f6773d36.0000000000000019:head v 94'440, mlcod=91'437)", 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:56.290516+0000", 2024-09-13T07:14:00.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.649750074, 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033261160000000001, 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290516+0000", 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290516+0000", 2024-09-13T07:14:00.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290515+0000", 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290521+0000", 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1850000000000001e-06 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290533+0000", 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1364e-05 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290540+0000", 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.52e-06 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290589+0000", 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-05 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.290611+0000", 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2526000000000001e-05 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.293805+0000", 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003193605 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:56.293842+0000", 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7347e-05 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.022 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:00.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:14:00.024 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:00.025+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:00.125 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:00.125+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:00.129 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:00.129 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:14:00.130 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:14:00.130 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:00.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:14:00.225 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:14:00.225+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:14:00.250 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:14:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T07:14:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T07:14:00.250 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:00.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:14:00.326 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:00.326+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850296747000002, 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:14:00.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:14:00.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850271433000003, 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:14:00.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850237067000002, 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:14:00.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850208242999997, 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850189946, 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.433 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850165140000001, 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:00.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:14:00.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850147266999997, 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850120926000002, 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:14:00.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:14:00.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850094204000001, 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850076618999999, 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850049980999998, 2024-09-13T07:14:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850032521000003, 2024-09-13T07:14:00.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.850005563000003, 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:00.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:14:00.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.849979359000002, 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.849961366999999, 2024-09-13T07:14:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.849869599999998, 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.8498473, 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.480 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.849816298999997, 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:14:00.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:14:00.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.849791543000002, 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:14:00.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:00.492 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.849773851000002, 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:14:00.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:00.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:00.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:00.497 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:00.498 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:14:00.499 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:14:00.499+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:14:00.599 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:00.599+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:00.601 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:00.601 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:00.601 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:00.601 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:00.610 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:14:00.700 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:00.701+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:00.712 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:00.712 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:00.712 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:00.712 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:00.712 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:00.721 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:14:00.801 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:00.802+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 74.608694482000004, 2024-09-13T07:14:00.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:14:00.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3:f56afaab:::rbd_object_map.117375db170a:head [read 0~112] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 46.478371223000003, 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0257515829999999, 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.831 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346593+0000", 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346603+0000", 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0037999999999999e-05 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346626+0000", 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3169000000000001e-05 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346638+0000", 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1712999999999999e-05 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346719+0000", 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0980000000000001e-05 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346730+0000", 2024-09-13T07:14:00.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0657e-05 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348732+0000", 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020022690000000001 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348743+0000", 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0431999999999999e-05 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.371984+0000", 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0232416790000001 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372159+0000", 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017503500000000001 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372347+0000", 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018757299999999999 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.836 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96143 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 2859008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.041808826999997, 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043884619, 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96143 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.837 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783094+0000", 2024-09-13T07:14:00.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999361 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783161+0000", 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6962e-05 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783169+0000", 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6620000000000005e-06 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783174+0000", 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4269999999999999e-06 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826848+0000", 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043674360000000002 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826943+0000", 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4864999999999994e-05 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.840 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.827042+0000", 2024-09-13T07:14:00.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9096000000000001e-05 2024-09-13T07:14:00.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.841 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.841 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96153 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3350528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039920471999999, 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044029318999999997, 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96153 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785048+0000", 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785054+0000", 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8680000000000004e-06 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785059+0000", 2024-09-13T07:14:00.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828723+0000", 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043664082999999999 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828800+0000", 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6612000000000004e-05 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829075+0000", 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027522899999999999 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96154 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039903222, 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044169422999999999, 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.846 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96154 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:00.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785065+0000", 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1789999999999998e-06 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785071+0000", 2024-09-13T07:14:00.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785075+0000", 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6310000000000002e-06 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829122+0000", 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044046253 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829175+0000", 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3631000000000002e-05 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829233+0000", 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7348e-05 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96155 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039886701999997, 2024-09-13T07:14:00.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044335547000000003, 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96155 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785079+0000", 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785082+0000", 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2060000000000001e-06 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785087+0000", 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785092+0000", 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5789999999999997e-06 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829271+0000", 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044178807 2024-09-13T07:14:00.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829346+0000", 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5072000000000002e-05 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829415+0000", 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9474000000000006e-05 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96156 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3497984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039870614999998, 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044473938999999997, 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96156 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785098+0000", 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0430000000000002e-06 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785103+0000", 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.446e-06 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785108+0000", 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:14:00.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829445+0000", 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044337623999999999 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829496+0000", 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0705000000000003e-05 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829570+0000", 2024-09-13T07:14:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3830000000000003e-05 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96157 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3547136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039854908000002, 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044623329000000003, 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.859 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96157 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785111+0000", 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785113+0000", 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785119+0000", 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6080000000000003e-06 2024-09-13T07:14:00.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785124+0000", 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6600000000000003e-06 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829591+0000", 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044467818999999999 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829668+0000", 2024-09-13T07:14:00.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6925e-05 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829735+0000", 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6631000000000006e-05 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96158 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039823347000002, 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044723374000000003, 2024-09-13T07:14:00.863 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96158 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785145+0000", 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785152+0000", 2024-09-13T07:14:00.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4049999999999997e-06 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785157+0000", 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.814e-06 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829772+0000", 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044615624 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829823+0000", 2024-09-13T07:14:00.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1131000000000002e-05 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829866+0000", 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3237e-05 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96159 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:14:00.867 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039805586999996, 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044827964999999997, 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96159 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785160+0000", 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785163+0000", 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2259999999999999e-06 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785168+0000", 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6219999999999997e-06 2024-09-13T07:14:00.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785173+0000", 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4989999999999996e-06 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829905+0000", 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044732582 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829949+0000", 2024-09-13T07:14:00.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3593000000000001e-05 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829989+0000", 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9919000000000002e-05 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96160 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3694592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039790046, 2024-09-13T07:14:00.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045000912999999997, 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96160 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785182+0000", 2024-09-13T07:14:00.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4550000000000003e-06 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785188+0000", 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4749999999999998e-06 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785193+0000", 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3109999999999996e-06 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830026+0000", 2024-09-13T07:14:00.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044832982 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830097+0000", 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1074000000000004e-05 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830177+0000", 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9949000000000003e-05 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96161 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039769049, 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045119541999999999, 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.877 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96161 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785199+0000", 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1399999999999998e-06 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785205+0000", 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6370000000000004e-06 2024-09-13T07:14:00.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785209+0000", 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830212+0000", 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045002535000000003 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830277+0000", 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5617999999999994e-05 2024-09-13T07:14:00.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830317+0000", 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9617999999999999e-05 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96162 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3792896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039752808999999, 2024-09-13T07:14:00.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196102000000002, 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96162 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785213+0000", 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785215+0000", 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.069e-06 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785221+0000", 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6980000000000003e-06 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785226+0000", 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5170000000000001e-06 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830332+0000", 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045106780999999999 2024-09-13T07:14:00.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830375+0000", 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2474e-05 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830410+0000", 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4909000000000001e-05 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96163 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.039731568999997, 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045359167999999998, 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96163 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785234+0000", 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785237+0000", 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7149999999999998e-06 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785243+0000", 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2349999999999996e-06 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785248+0000", 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.78e-06 2024-09-13T07:14:00.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830445+0000", 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196419000000002 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830514+0000", 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9029999999999995e-05 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830594+0000", 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0520999999999997e-05 2024-09-13T07:14:00.889 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96165 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.038057692999999, 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044000064999999998, 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.890 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96165 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786908+0000", 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786911+0000", 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786916+0000", 2024-09-13T07:14:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.84e-06 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786921+0000", 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830763+0000", 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043841106999999997 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830834+0000", 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1304999999999996e-05 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830909+0000", 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4987999999999994e-05 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96166 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.038040623000001, 2024-09-13T07:14:00.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044109809, 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96166 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786925+0000", 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786928+0000", 2024-09-13T07:14:00.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0820000000000001e-06 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786933+0000", 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786938+0000", 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830944+0000", 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044005374999999999 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830993+0000", 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8900000000000003e-05 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831036+0000", 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3139000000000002e-05 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.898 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96167 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.038024301999997, 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044213595000000001, 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96167 2024-09-13T07:14:00.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786944+0000", 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3829999999999999e-06 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786950+0000", 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:14:00.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786955+0000", 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6059999999999996e-06 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831062+0000", 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044107252999999999 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831101+0000", 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9175999999999999e-05 2024-09-13T07:14:00.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831156+0000", 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4586999999999998e-05 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96168 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.038008167999998, 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044324181999999997, 2024-09-13T07:14:00.903 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96168 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786960+0000", 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0789999999999999e-06 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786966+0000", 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6330000000000001e-06 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786970+0000", 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831185+0000", 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044215245 2024-09-13T07:14:00.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831231+0000", 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5952000000000003e-05 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831282+0000", 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1258999999999998e-05 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.907 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96169 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4136960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.037992778000003, 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044443489000000003, 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96169 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786973+0000", 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786975+0000", 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786981+0000", 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4389999999999999e-06 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786985+0000", 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5929999999999999e-06 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831321+0000", 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044336028 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831380+0000", 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8329999999999999e-05 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831417+0000", 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7361999999999999e-05 2024-09-13T07:14:00.911 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96170 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.037968313999997, 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523786000000003, 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:00.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96170 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787001+0000", 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.932e-06 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787007+0000", 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1739999999999997e-06 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787012+0000", 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.899e-06 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831437+0000", 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044425112000000003 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831480+0000", 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2864000000000002e-05 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:00.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831522+0000", 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2366000000000001e-05 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:00.916 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:00.918 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:14:00.918 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:00.918+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:01.018 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:01.019+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:01.061 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:01.061 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:01.061 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:01.061 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:01.069 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:14:01.119 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:01.119+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:01.211 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:01.211 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:01.211 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:01.211 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:01.212 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:01.220 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:14:01.220 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:01.220+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:01.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:14:01.320 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:01.321+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:01.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:14:01.373 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:01.373 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.710106660999998, 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:14:01.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.710069009999998, 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:14:01.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:14:01.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:14:01.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.710049783000002, 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:14:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 56.833976370999999, 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.387 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:01.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:14:01.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96967 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.525039049, 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028701085000000001, 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96967 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848739+0000", 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848741+0000", 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.153e-06 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848743+0000", 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.723e-06 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848746+0000", 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.858e-06 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877369+0000", 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028623282999999999 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877406+0000", 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6139000000000001e-05 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877441+0000", 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151000000000003e-05 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96968 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.525030487999999, 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028858604999999999, 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96968 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848749+0000", 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.091e-06 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848752+0000", 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7769999999999998e-06 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848754+0000", 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2869999999999999e-06 2024-09-13T07:14:01.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877468+0000", 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028714185 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877541+0000", 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2927000000000002e-05 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877607+0000", 2024-09-13T07:14:01.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5560000000000002e-05 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96969 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.52502234, 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028944845, 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.402 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96969 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848757+0000", 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-13T07:14:01.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848761+0000", 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848763+0000", 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.311e-06 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877630+0000", 2024-09-13T07:14:01.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866326000000001 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877666+0000", 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6127000000000002e-05 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877701+0000", 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5413000000000001e-05 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.406 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96970 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.525013348999998, 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029100663999999998, 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96970 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.407 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848766+0000", 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.097e-06 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848769+0000", 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848772+0000", 2024-09-13T07:14:01.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4269999999999998e-06 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877729+0000", 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028957488 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877800+0000", 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1113999999999997e-05 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877866+0000", 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5882000000000004e-05 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96971 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.525005219000001, 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029181689, 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.411 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96971 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:01.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848774+0000", 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1209999999999999e-06 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848777+0000", 2024-09-13T07:14:01.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.926e-06 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848780+0000", 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877884+0000", 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029104076999999999 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877919+0000", 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5655999999999999e-05 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877955+0000", 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5782999999999999e-05 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.415 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96972 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524997028000001, 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029338843999999999, 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96972 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848781+0000", 2024-09-13T07:14:01.417 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848783+0000", 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1659999999999999e-06 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848785+0000", 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.818e-06 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848788+0000", 2024-09-13T07:14:01.418 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4569999999999999e-06 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877984+0000", 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029196203 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878055+0000", 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0622000000000002e-05 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.419 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878120+0000", 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5816999999999997e-05 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96973 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524971858000001, 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029407188000000001, 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.420 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96973 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848806+0000", 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848808+0000", 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7039999999999999e-06 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.422 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848813+0000", 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6360000000000002e-06 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848815+0000", 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5799999999999999e-06 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878143+0000", 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029327678999999999 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.423 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878179+0000", 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5587000000000003e-05 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878214+0000", 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5456000000000001e-05 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.424 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96974 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524960559, 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029557028999999999, 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96974 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.425 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.426 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848821+0000", 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5140000000000002e-06 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848825+0000", 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848827+0000", 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.39e-06 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878240+0000", 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029413174 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878310+0000", 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9813000000000004e-05 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.428 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878375+0000", 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5122000000000004e-05 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96975 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524949518, 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029640264, 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.429 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96975 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.431 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848830+0000", 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.235e-06 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848833+0000", 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.796e-06 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848835+0000", 2024-09-13T07:14:01.432 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.402e-06 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878397+0000", 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029561548999999999 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878434+0000", 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6721000000000003e-05 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878469+0000", 2024-09-13T07:14:01.433 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5855999999999997e-05 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96976 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524941398000003, 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029795095000000001, 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.434 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96976 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.435 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848838+0000", 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.156e-06 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848841+0000", 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9110000000000002e-06 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.436 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848843+0000", 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878496+0000", 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029652371 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878567+0000", 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0784000000000005e-05 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.437 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878632+0000", 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5783000000000004e-05 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96977 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524933267000002, 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029869717, 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.438 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96977 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.439 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848846+0000", 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1000000000000001e-06 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848849+0000", 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.835e-06 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.440 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848851+0000", 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878643+0000", 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029791405 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.441 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878678+0000", 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5139999999999999e-05 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878715+0000", 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7119000000000002e-05 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96978 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:01.442 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524925250000003, 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030029390999999999, 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96978 2024-09-13T07:14:01.443 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.444 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848854+0000", 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1039999999999999e-06 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848857+0000", 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8399999999999999e-06 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.445 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848859+0000", 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878742+0000", 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029882140000000001 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878813+0000", 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1771000000000001e-05 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.446 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878883+0000", 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9457999999999998e-05 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96979 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524917318, 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030106193999999999, 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.447 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96979 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:01.448 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.449 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848862+0000", 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.234e-06 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848865+0000", 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.836e-06 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848868+0000", 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878895+0000", 2024-09-13T07:14:01.450 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030027642 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878931+0000", 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5624999999999999e-05 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.451 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878968+0000", 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6720000000000001e-05 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96980 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524909159000003, 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030267181000000001, 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.452 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96980 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.453 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848869+0000", 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.088e-06 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848873+0000", 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9019999999999999e-06 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.454 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848875+0000", 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1950000000000002e-06 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878996+0000", 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03012068 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879067+0000", 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1100999999999996e-05 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879137+0000", 2024-09-13T07:14:01.455 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9424999999999993e-05 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96981 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524895338999997, 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030344368, 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.456 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96981 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.457 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848885+0000", 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848888+0000", 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.458 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848891+0000", 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.554e-06 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879159+0000", 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030268155000000001 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879192+0000", 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.459 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879228+0000", 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6146999999999998e-05 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96982 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 33.524886074000001, 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030488562, 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:01.460 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96982 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:14:01.461 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848892+0000", 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848894+0000", 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2079999999999999e-06 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848896+0000", 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8679999999999999e-06 2024-09-13T07:14:01.462 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848899+0000", 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879246+0000", 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030346838000000001 2024-09-13T07:14:01.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879317+0000", 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1254000000000006e-05 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879381+0000", 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4331000000000005e-05 2024-09-13T07:14:01.464 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.465 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:01.465 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:01.465 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:01.467 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:14:01.467 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:01.467+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:01.480 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:14:01.481 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:14:01.497 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:14:01.498 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:14:01.513 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:14:01.513 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:14:01.529 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:14:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:14:01.545 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:14:01.546 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:14:01.563 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:14:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:14:01.567 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:01.568+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:01.579 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:14:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:14:01.595 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:14:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:14:01.613 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:01.613 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:01.621 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:14:01.669 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:14:01.668+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:14:01.763 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:01.763 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:01.770 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:01.769+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:01.772 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:14:01.870 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:01.871+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:01.874 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:01.874 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:01.884 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:14:01.972 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:01.972+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:01.986 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:01.986 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:01.995 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:14:02.073 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:14:02.073+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:02.097 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:02.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:14:02.174 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:02.175+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:02.208 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:02.209 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:02.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:14:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:14:02.302 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:02.302+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:02.349 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:02.350 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:02.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:14:02.376 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:02.377+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:02.476 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:14:02.476+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:14:02.506 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:02.506 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:02.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:14:02.560 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:14:02.560 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:14:02.560 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-13T07:14:02.560 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:14:02.560 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:14:02.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-13T07:14:02.577 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:02.578+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:02.662 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:02.663 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:02.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:14:02.677 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:02.678+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:02.706 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "" 2024-09-13T07:14:02.706 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:02.779 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:02.780+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:02.779 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:02.779 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:02.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:14:02.880 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:02.881+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:02.894 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:02.894 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:02.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:14:02.981 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:02.981+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:03.010 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:03.010 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:03.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:14:03.082 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:03.083+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:03.172 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:03.173 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:03.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:14:03.183 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:03.184+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:03.285 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:03.286+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:03.300 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:03.301 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:03.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:14:03.386 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:03.387+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:03.416 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:03.417 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:03.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:14:03.487 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:14:03.487+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:14:03.533 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:03.533 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:03.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:14:03.588 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:03.589+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:03.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:14:03.689 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:03.689 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:03.690 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:03.689+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:03.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:14:03.748 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:14:03.749 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:14:03.766 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:14:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:14:03.783 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:14:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:14:03.790 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:03.790+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:03.801 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:14:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:14:03.818 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:14:03.818 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:14:03.851 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:14:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:14:03.869 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51884 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3723264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308347412000003, 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074226049000000002, 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51884 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561407+0000", 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561406+0000", 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561417+0000", 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561423+0000", 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561428+0000", 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635451+0000", 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074022976000000004 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635527+0000", 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:14:03.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635633+0000", 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105696 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51885 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308322097999998, 2024-09-13T07:14:03.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074415672000000002, 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51885 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561432+0000", 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561434+0000", 2024-09-13T07:14:03.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561440+0000", 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561444+0000", 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3490000000000002e-06 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635661+0000", 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074216775999999998 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635745+0000", 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4245000000000004e-05 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635848+0000", 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102513 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51886 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308287731999997, 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074677684999999994, 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51886 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561467+0000", 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561466+0000", 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561469+0000", 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561477+0000", 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7170000000000008e-06 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561482+0000", 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2340000000000002e-06 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.635878+0000", 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074395955 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636027+0000", 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014876000000000001 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636144+0000", 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117656 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51887 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308258907999999, 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074883837999999994, 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51887 2024-09-13T07:14:03.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561495+0000", 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561486+0000", 2024-09-13T07:14:03.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999909 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561498+0000", 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1392999999999999e-05 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561504+0000", 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561509+0000", 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636177+0000", 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074668044000000003 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636262+0000", 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4388999999999995e-05 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636379+0000", 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011761599999999999 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51888 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308240611000002, 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075099979999999997, 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51888 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561514+0000", 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561513+0000", 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561515+0000", 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0930000000000001e-06 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561521+0000", 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7080000000000002e-06 2024-09-13T07:14:03.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561533+0000", 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636409+0000", 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074875947999999998 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636502+0000", 2024-09-13T07:14:03.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636614+0000", 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111699 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51889 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3846144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308215805000003, 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075296270999999998, 2024-09-13T07:14:03.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51889 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561538+0000", 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561541+0000", 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6110000000000001e-06 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561547+0000", 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561552+0000", 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999996e-06 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636644+0000", 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075091896000000005 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636730+0000", 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6025000000000004e-05 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636835+0000", 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104625 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51890 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:03.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308197931999999, 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075516469000000003, 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51890 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561556+0000", 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561558+0000", 2024-09-13T07:14:03.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561572+0000", 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4096e-05 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561577+0000", 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636865+0000", 2024-09-13T07:14:03.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075287729999999997 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.636961+0000", 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6156000000000006e-05 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637073+0000", 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011152 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51891 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3895296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308171590999997, 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075677415999999997, 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51891 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561583+0000", 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561582+0000", 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561585+0000", 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7199999999999998e-06 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561591+0000", 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1840000000000004e-06 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561596+0000", 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7230000000000004e-06 2024-09-13T07:14:03.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637099+0000", 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075503411000000006 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637172+0000", 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3090999999999995e-05 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637260+0000", 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7658999999999995e-05 2024-09-13T07:14:03.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51892 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3919872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308144869000003, 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075862382000000006, 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51892 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561609+0000", 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561612+0000", 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561618+0000", 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561623+0000", 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637286+0000", 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075663414999999998 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637360+0000", 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3996000000000001e-05 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637472+0000", 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111512 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51893 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3944448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308127284000001, 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076109105999999996, 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51893 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561627+0000", 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561629+0000", 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561644+0000", 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4676e-05 2024-09-13T07:14:03.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561649+0000", 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637517+0000", 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075868325 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637605+0000", 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8474999999999996e-05 2024-09-13T07:14:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637736+0000", 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130772 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51894 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3969024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308100646, 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076312130000000006, 2024-09-13T07:14:03.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51894 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561654+0000", 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561653+0000", 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561656+0000", 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561662+0000", 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3119999999999999e-06 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561667+0000", 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637766+0000", 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076099397999999999 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637850+0000", 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3920999999999998e-05 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637966+0000", 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115699 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51895 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308083185999998, 2024-09-13T07:14:03.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076501066000000006, 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51895 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561671+0000", 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561681+0000", 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0278999999999999e-05 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561688+0000", 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561693+0000", 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.637993+0000", 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076300244000000003 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638071+0000", 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7554999999999999e-05 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638172+0000", 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101523 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51896 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4018176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:03.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308056227999998, 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076695484999999994, 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51896 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561698+0000", 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561700+0000", 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.435e-06 2024-09-13T07:14:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561706+0000", 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561711+0000", 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638198+0000", 2024-09-13T07:14:03.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076487335000000004 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638279+0000", 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129000000000005e-05 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638394+0000", 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115017 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51897 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308030023999997, 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076989688000000001, 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51897 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561724+0000", 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561726+0000", 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5490000000000001e-06 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561733+0000", 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561738+0000", 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:14:03.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638457+0000", 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076719190000000007 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638552+0000", 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638714+0000", 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016192800000000001 2024-09-13T07:14:03.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51898 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4067328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.308012032000001, 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077213808999999994, 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51898 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561742+0000", 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561769+0000", 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7404000000000001e-05 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561777+0000", 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561783+0000", 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638744+0000", 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076960799999999996 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638832+0000", 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7854000000000003e-05 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.638956+0000", 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012427200000000001 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51899 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4091904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.307920265, 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078381047999999995, 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51899 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561834+0000", 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561833+0000", 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561837+0000", 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4410000000000002e-06 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561845+0000", 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561851+0000", 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9830000000000001e-06 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.639947+0000", 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078095824999999994 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640046+0000", 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8479000000000004e-05 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640215+0000", 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169502 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51900 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.307897965000002, 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078628167999999998, 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51900 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:03.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561856+0000", 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561858+0000", 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561872+0000", 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3862999999999999e-05 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561878+0000", 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640245+0000", 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078367634000000005 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640341+0000", 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5089999999999999e-05 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640484+0000", 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143914 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51901 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4141056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.307866963999999, 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078829518000000001, 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51901 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561887+0000", 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561890+0000", 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.439e-06 2024-09-13T07:14:03.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561902+0000", 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1593999999999999e-05 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561907+0000", 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4129999999999997e-06 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640515+0000", 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078607233999999998 2024-09-13T07:14:03.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640600+0000", 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5678000000000002e-05 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640717+0000", 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011661 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51902 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4165632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.307842207999997, 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078948409999999997, 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51902 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561912+0000", 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561914+0000", 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561921+0000", 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2419999999999997e-06 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561925+0000", 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640734+0000", 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078808248999999997 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640776+0000", 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2688000000000002e-05 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640860+0000", 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4142999999999997e-05 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4467.0:51903 3.15 3:adbc3af2:::rbd_data.1173fee7d7bf.0000000000000007:15 [read 4190208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.307824515999997, 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079088182000000007, 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4467", 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51903 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561930+0000", 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561929+0000", 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:03.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561932+0000", 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561938+0000", 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.003e-06 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.561942+0000", 2024-09-13T07:14:03.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640874+0000", 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078931847999999999 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.640932+0000", 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7669e-05 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T07:13:04.641018+0000", 2024-09-13T07:14:03.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5977999999999996e-05 2024-09-13T07:14:03.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:03.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:03.954 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:03.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:14:03.955 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:14:03.956 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:14:03.956 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:14:03.957 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:14:03.957 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:03.956+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:04.055 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:04.056+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:04.077 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:04.077 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:04.077 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:04.077 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:04.087 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:14:04.157 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:04.158+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:04.193 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:04.193 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:04.193 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:04.193 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:04.193 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:04.202 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:14:04.257 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:04.258+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:04.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:14:04.310 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:04.310 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:04.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:04.310 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:04.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.310 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:6114 3.f 3:f0d5ca53:::rbd_data.11734b163e95.000000000000005f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 232217~29927] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:14:04.310 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 78.094147660999994, 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.011190153, 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 6114 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.311 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216272+0000", 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216271+0000", 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216355+0000", 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.4196000000000005e-05 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216367+0000", 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2299e-05 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216376+0000", 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4129999999999995e-06 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216411+0000", 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4864999999999999e-05 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:46.216419+0000", 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4540000000000001e-06 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224248+0000", 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0078290700000001 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.224378+0000", 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00012997099999999999 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227386+0000", 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.003008527 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227420+0000", 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.392e-05 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:47.227462+0000", 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1912999999999997e-05 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.316 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3:f56afaab:::rbd_object_map.117375db170a:head [read 0~112] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 49.963824402, 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0257515829999999, 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346595+0000", 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346593+0000", 2024-09-13T07:14:04.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346603+0000", 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0037999999999999e-05 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346626+0000", 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3169000000000001e-05 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346638+0000", 2024-09-13T07:14:04.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1712999999999999e-05 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346719+0000", 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0980000000000001e-05 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.346730+0000", 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0657e-05 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.320 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348732+0000", 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0020022690000000001 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.348743+0000", 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0431999999999999e-05 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.371984+0000", 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0232416790000001 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.321 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372159+0000", 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00017503500000000001 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:15.372347+0000", 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018757299999999999 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.322 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96143 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 2859008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.527262006000001, 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043884619, 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96143 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.323 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783158+0000", 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783094+0000", 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999361 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783161+0000", 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6962e-05 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783169+0000", 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6620000000000005e-06 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.783174+0000", 2024-09-13T07:14:04.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4269999999999999e-06 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826848+0000", 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043674360000000002 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.826943+0000", 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4864999999999994e-05 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.827042+0000", 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.9096000000000001e-05 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96153 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3350528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525373651000002, 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044029318999999997, 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.327 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96153 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785046+0000", 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785048+0000", 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0559999999999999e-06 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785054+0000", 2024-09-13T07:14:04.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8680000000000004e-06 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785059+0000", 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828723+0000", 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043664082999999999 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.828800+0000", 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6612000000000004e-05 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829075+0000", 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00027522899999999999 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96154 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525356401000003, 2024-09-13T07:14:04.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044169422999999999, 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96154 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:04.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785063+0000", 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785065+0000", 2024-09-13T07:14:04.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1789999999999998e-06 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785071+0000", 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785075+0000", 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6310000000000002e-06 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829122+0000", 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044046253 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829175+0000", 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3631000000000002e-05 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829233+0000", 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7348e-05 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.335 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96155 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525339881000001, 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044335547000000003, 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.336 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96155 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785080+0000", 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785079+0000", 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785082+0000", 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2060000000000001e-06 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785087+0000", 2024-09-13T07:14:04.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785092+0000", 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5789999999999997e-06 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829271+0000", 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044178807 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829346+0000", 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5072000000000002e-05 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829415+0000", 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9474000000000006e-05 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96156 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3497984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.340 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525323794000002, 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044473938999999997, 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96156 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785096+0000", 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785098+0000", 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0430000000000002e-06 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785103+0000", 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.446e-06 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785108+0000", 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829445+0000", 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044337623999999999 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829496+0000", 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0705000000000003e-05 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829570+0000", 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3830000000000003e-05 2024-09-13T07:14:04.344 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96157 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3547136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525308086999999, 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044623329000000003, 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.345 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96157 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785112+0000", 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785111+0000", 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785113+0000", 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0269999999999998e-06 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785119+0000", 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6080000000000003e-06 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785124+0000", 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6600000000000003e-06 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829591+0000", 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044467818999999999 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.348 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829668+0000", 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6925e-05 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829735+0000", 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6631000000000006e-05 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96158 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:04.349 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525276525999999, 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044723374000000003, 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96158 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:04.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785143+0000", 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785145+0000", 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785152+0000", 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4049999999999997e-06 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785157+0000", 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.814e-06 2024-09-13T07:14:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829772+0000", 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044615624 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829823+0000", 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1131000000000002e-05 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829866+0000", 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3237e-05 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96159 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525258766, 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044827964999999997, 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.354 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96159 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785161+0000", 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785160+0000", 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785163+0000", 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2259999999999999e-06 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785168+0000", 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6219999999999997e-06 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785173+0000", 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4989999999999996e-06 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829905+0000", 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044732582 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829949+0000", 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3593000000000001e-05 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.829989+0000", 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9919000000000002e-05 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96160 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3694592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.358 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525243224999997, 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045000912999999997, 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96160 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:04.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785176+0000", 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785182+0000", 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4550000000000003e-06 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785188+0000", 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4749999999999998e-06 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785193+0000", 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3109999999999996e-06 2024-09-13T07:14:04.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830026+0000", 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044832982 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830097+0000", 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1074000000000004e-05 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830177+0000", 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9949000000000003e-05 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96161 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525222227999997, 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045119541999999999, 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.363 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96161 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:04.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785197+0000", 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785199+0000", 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1399999999999998e-06 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.365 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785205+0000", 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6370000000000004e-06 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785209+0000", 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830212+0000", 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045002535000000003 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830277+0000", 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5617999999999994e-05 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830317+0000", 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9617999999999999e-05 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96162 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3792896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525205988000003, 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196102000000002, 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96162 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.368 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785214+0000", 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785213+0000", 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785215+0000", 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.069e-06 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785221+0000", 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6980000000000003e-06 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785226+0000", 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5170000000000001e-06 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830332+0000", 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045106780999999999 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830375+0000", 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2474e-05 2024-09-13T07:14:04.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830410+0000", 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4909000000000001e-05 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96163 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.525184748000001, 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045359167999999998, 2024-09-13T07:14:04.372 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96163 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785235+0000", 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785234+0000", 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785237+0000", 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7149999999999998e-06 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785243+0000", 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2349999999999996e-06 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.785248+0000", 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.78e-06 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830445+0000", 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.045196419000000002 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830514+0000", 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9029999999999995e-05 2024-09-13T07:14:04.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830594+0000", 2024-09-13T07:14:04.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0520999999999997e-05 2024-09-13T07:14:04.377 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.377 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.377 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96165 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.523510872000003, 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044000064999999998, 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96165 2024-09-13T07:14:04.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786909+0000", 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786908+0000", 2024-09-13T07:14:04.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786911+0000", 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.379e-06 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786916+0000", 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.84e-06 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786921+0000", 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9699999999999998e-06 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830763+0000", 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.043841106999999997 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830834+0000", 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1304999999999996e-05 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830909+0000", 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4987999999999994e-05 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.382 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96166 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.523493801999997, 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044109809, 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96166 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786926+0000", 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786925+0000", 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786928+0000", 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0820000000000001e-06 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786933+0000", 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786938+0000", 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830944+0000", 2024-09-13T07:14:04.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044005374999999999 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.830993+0000", 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8900000000000003e-05 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831036+0000", 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3139000000000002e-05 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.387 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96167 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.523477481, 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044213595000000001, 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96167 2024-09-13T07:14:04.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786942+0000", 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786944+0000", 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3829999999999999e-06 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786950+0000", 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9109999999999998e-06 2024-09-13T07:14:04.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786955+0000", 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6059999999999996e-06 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831062+0000", 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044107252999999999 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831101+0000", 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.9175999999999999e-05 2024-09-13T07:14:04.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831156+0000", 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4586999999999998e-05 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96168 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.523461347000001, 2024-09-13T07:14:04.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044324181999999997, 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96168 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786958+0000", 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786960+0000", 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0789999999999999e-06 2024-09-13T07:14:04.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786966+0000", 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6330000000000001e-06 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786970+0000", 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3390000000000003e-06 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831185+0000", 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044215245 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831231+0000", 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5952000000000003e-05 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831282+0000", 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1258999999999998e-05 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.396 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96169 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4136960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.523445957, 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044443489000000003, 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96169 2024-09-13T07:14:04.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786974+0000", 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786973+0000", 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786975+0000", 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0770000000000002e-06 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786981+0000", 2024-09-13T07:14:04.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4389999999999999e-06 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786985+0000", 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5929999999999999e-06 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831321+0000", 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044336028 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831380+0000", 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8329999999999999e-05 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831417+0000", 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7361999999999999e-05 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96170 3.9 3:91b2c9e1:::rbd_data.117390733e87.000000000000001d:21 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 36.523421493000001, 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044523786000000003, 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96170 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.786998+0000", 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787001+0000", 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.932e-06 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787007+0000", 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1739999999999997e-06 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.787012+0000", 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.899e-06 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831437+0000", 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.044425112000000003 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831480+0000", 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2864000000000002e-05 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.831522+0000", 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2366000000000001e-05 2024-09-13T07:14:04.405 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.406 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.406 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.406 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.406 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.406 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:04.407 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:14:04.408 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:04.408+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:04.508 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:04.509+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:04.551 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:04.552 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:04.552 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:04.552 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:04.560 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:14:04.644 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:04.645+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:04.703 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:04.703 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:04.703 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:04.703 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:04.703 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:04.710 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:04.711+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:04.712 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:14:04.812 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:04.812+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:04.856 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:04.856 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36794 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13541376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 63.192876841, 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028770805999999999, 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36794 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663672+0000", 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663674+0000", 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2220000000000001e-06 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663679+0000", 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3900000000000001e-06 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663684+0000", 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.313e-06 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692371+0000", 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028687481000000001 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692407+0000", 2024-09-13T07:14:04.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6368999999999997e-05 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692443+0000", 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5435000000000002e-05 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36795 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13565952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 63.192839190000001, 2024-09-13T07:14:04.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028831221000000001, 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36795 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663710+0000", 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663709+0000", 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663712+0000", 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6560000000000001e-06 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663720+0000", 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:14:04.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663725+0000", 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9609999999999996e-06 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692465+0000", 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028740628000000001 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692507+0000", 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1467999999999997e-05 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692541+0000", 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4230000000000003e-05 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:36796 3.5 3:a739230a:::rbd_data.1173fee7d7bf.000000000000000a:15 [read 13590528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:14:04.866 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 63.192819962999998, 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028902550999999999, 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 36796 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663729+0000", 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663728+0000", 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663731+0000", 2024-09-13T07:14:04.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0980000000000001e-06 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663736+0000", 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1410000000000004e-06 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.663740+0000", 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.284e-06 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692561+0000", 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028821513999999999 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692598+0000", 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6542999999999999e-05 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:01.692631+0000", 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3411999999999998e-05 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:51631 3.5 3:a6d4a28a:::rbd_data.1173a2795123.0000000000000013:head [delete] snapc 16=[16] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 60.316746551000001, 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030330816, 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 51631 2024-09-13T07:14:04.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539802+0000", 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539804+0000", 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1880000000000001e-06 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539808+0000", 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4850000000000001e-06 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.539810+0000", 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540689+0000", 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00087844400000000003 2024-09-13T07:14:04.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540755+0000", 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6305999999999998e-05 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 3", 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.540942+0000", 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00018681399999999999 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.541753+0000", 2024-09-13T07:14:04.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00081100499999999995 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570087+0000", 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028334333 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570113+0000", 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6061999999999999e-05 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:04.570133+0000", 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9828999999999999e-05 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96967 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007809229000003, 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028701085000000001, 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96967 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848740+0000", 2024-09-13T07:14:04.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848739+0000", 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848741+0000", 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.153e-06 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848743+0000", 2024-09-13T07:14:04.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.723e-06 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848746+0000", 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.858e-06 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877369+0000", 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028623282999999999 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877406+0000", 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6139000000000001e-05 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877441+0000", 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5151000000000003e-05 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96968 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.881 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007800668000002, 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028858604999999999, 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96968 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:04.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848748+0000", 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848749+0000", 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.091e-06 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848752+0000", 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7769999999999998e-06 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848754+0000", 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2869999999999999e-06 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877468+0000", 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028714185 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877541+0000", 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2927000000000002e-05 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877607+0000", 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5560000000000002e-05 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.885 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96969 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007792520000002, 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028944845, 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.886 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96969 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848756+0000", 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848757+0000", 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1119999999999999e-06 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848761+0000", 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848763+0000", 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.311e-06 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877630+0000", 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866326000000001 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877666+0000", 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6127000000000002e-05 2024-09-13T07:14:04.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877701+0000", 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5413000000000001e-05 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96970 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007783529000001, 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029100663999999998, 2024-09-13T07:14:04.890 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96970 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848765+0000", 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848766+0000", 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.097e-06 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848769+0000", 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848772+0000", 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4269999999999998e-06 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877729+0000", 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028957488 2024-09-13T07:14:04.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877800+0000", 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1113999999999997e-05 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877866+0000", 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5882000000000004e-05 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96971 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007775399000003, 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029181689, 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96971 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848773+0000", 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848774+0000", 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1209999999999999e-06 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848777+0000", 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.926e-06 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848780+0000", 2024-09-13T07:14:04.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877884+0000", 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029104076999999999 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877919+0000", 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5655999999999999e-05 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877955+0000", 2024-09-13T07:14:04.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5782999999999999e-05 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96972 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007767207999997, 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029338843999999999, 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96972 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848782+0000", 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848781+0000", 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848783+0000", 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1659999999999999e-06 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848785+0000", 2024-09-13T07:14:04.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.818e-06 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848788+0000", 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4569999999999999e-06 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.877984+0000", 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029196203 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878055+0000", 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0622000000000002e-05 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878120+0000", 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5816999999999997e-05 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96973 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.903 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007742038000004, 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029407188000000001, 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96973 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848807+0000", 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848806+0000", 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848808+0000", 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.7039999999999999e-06 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848813+0000", 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.6360000000000002e-06 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848815+0000", 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5799999999999999e-06 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878143+0000", 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029327678999999999 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878179+0000", 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5587000000000003e-05 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878214+0000", 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5456000000000001e-05 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96974 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007730739000003, 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029557028999999999, 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96974 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.908 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848818+0000", 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848821+0000", 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5140000000000002e-06 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848825+0000", 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848827+0000", 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.39e-06 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878240+0000", 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029413174 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878310+0000", 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9813000000000004e-05 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878375+0000", 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5122000000000004e-05 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96975 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:04.912 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007719698000002, 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029640264, 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96975 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848829+0000", 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848830+0000", 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.235e-06 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848833+0000", 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.796e-06 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848835+0000", 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.402e-06 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878397+0000", 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029561548999999999 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878434+0000", 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6721000000000003e-05 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878469+0000", 2024-09-13T07:14:04.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5855999999999997e-05 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96976 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007711577999999, 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029795095000000001, 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.917 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96976 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848837+0000", 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848838+0000", 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.156e-06 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848841+0000", 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9110000000000002e-06 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848843+0000", 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2960000000000001e-06 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878496+0000", 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029652371 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878567+0000", 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0784000000000005e-05 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878632+0000", 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5783000000000004e-05 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96977 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007703446999997, 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029869717, 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.921 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96977 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848845+0000", 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848846+0000", 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1000000000000001e-06 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848849+0000", 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.835e-06 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848851+0000", 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878643+0000", 2024-09-13T07:14:04.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029791405 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878678+0000", 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5139999999999999e-05 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878715+0000", 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.7119000000000002e-05 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.925 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96978 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007695429999998, 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030029390999999999, 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96978 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.926 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848853+0000", 2024-09-13T07:14:04.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848854+0000", 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1039999999999999e-06 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848857+0000", 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8399999999999999e-06 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848859+0000", 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878742+0000", 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029882140000000001 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878813+0000", 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1771000000000001e-05 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878883+0000", 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9457999999999998e-05 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96979 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007687498000003, 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030106193999999999, 2024-09-13T07:14:04.930 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96979 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848861+0000", 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848862+0000", 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.234e-06 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.932 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848865+0000", 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.836e-06 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848868+0000", 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878895+0000", 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030027642 2024-09-13T07:14:04.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878931+0000", 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.5624999999999999e-05 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878968+0000", 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6720000000000001e-05 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96980 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007679338999999, 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030267181000000001, 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96980 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848869+0000", 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848870+0000", 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.088e-06 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848873+0000", 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9019999999999999e-06 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848875+0000", 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1950000000000002e-06 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.937 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.878996+0000", 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.03012068 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879067+0000", 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1100999999999996e-05 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879137+0000", 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9424999999999993e-05 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.938 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96981 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007665519, 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030344368, 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96981 2024-09-13T07:14:04.939 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848883+0000", 2024-09-13T07:14:04.940 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848885+0000", 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6700000000000001e-06 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848888+0000", 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.941 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848891+0000", 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.554e-06 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879159+0000", 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030268155000000001 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879192+0000", 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2814999999999997e-05 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.942 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879228+0000", 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6146999999999998e-05 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:96982 3.1c 3:3acf213d:::rbd_data.117390733e87.000000000000001e:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.007656253999997, 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030488562, 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:04.943 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 96982 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:04.944 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848893+0000", 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848892+0000", 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848894+0000", 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2079999999999999e-06 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.945 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848896+0000", 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8679999999999999e-06 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.848899+0000", 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2879999999999999e-06 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879246+0000", 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.030346838000000001 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.946 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879317+0000", 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1254000000000006e-05 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.879381+0000", 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4331000000000005e-05 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:04.947 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:04.949 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:14:04.950 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:14:04.949+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:14:05.050 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:14:05.050+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:14:05.094 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:05.094 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:05.094 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:05.094 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:05.102 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:14:05.151 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:05.152+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:05.204 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:05.204 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:05.205 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:05.205 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:05.205 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:05.213 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:14:05.252 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:14:05.253+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:14:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:14:05.323 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 87.285979838000003, 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:14:05.324 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.326 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:14:05.327 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:14:05.328 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.739070966, 2024-09-13T07:14:05.329 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.330 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.331 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:14:05.332 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:14:05.333 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.334 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88519 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.737913132000003, 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017910447999999999, 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.335 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88519 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.336 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585257+0000", 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585267+0000", 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0635000000000001e-05 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585289+0000", 2024-09-13T07:14:05.337 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1846999999999999e-05 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585299+0000", 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4949999999999997e-06 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585347+0000", 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7672e-05 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.338 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585500+0000", 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015363700000000001 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.588596+0000", 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030953669999999999 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603068+0000", 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014472063 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.339 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603106+0000", 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8435e-05 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603142+0000", 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6488e-05 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.340 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603169+0000", 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6959000000000001e-05 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:89021 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 40.793436219, 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02102269, 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.341 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 89021 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.342 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529734+0000", 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529742+0000", 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8290000000000008e-06 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.343 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529757+0000", 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4868999999999999e-05 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529763+0000", 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.93e-06 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529803+0000", 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0404999999999997e-05 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.344 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529809+0000", 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531015+0000", 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012062049999999999 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531057+0000", 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1921000000000001e-05 2024-09-13T07:14:05.345 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.534526+0000", 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034694909999999999 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550683+0000", 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016157169999999998 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550697+0000", 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3351e-05 2024-09-13T07:14:05.346 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550731+0000", 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3997e-05 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550759+0000", 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7861e-05 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.347 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95026 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.627422152999998, 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014972104999999999, 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95026 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.348 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.349 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695752+0000", 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.283e-06 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695758+0000", 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695762+0000", 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.350 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710581+0000", 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014818934000000001 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710653+0000", 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2457000000000001e-05 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710722+0000", 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8633999999999994e-05 2024-09-13T07:14:05.351 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95027 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.627406043999997, 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015079646, 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.352 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95027 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.353 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695768+0000", 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695773+0000", 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.592e-06 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.354 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695778+0000", 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710749+0000", 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014970809 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710805+0000", 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6149000000000001e-05 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.355 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710846+0000", 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0840999999999997e-05 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95028 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.627390347999999, 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015231216000000001, 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.356 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95028 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:14:05.357 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695781+0000", 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695783+0000", 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9250000000000002e-06 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.358 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695789+0000", 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.835e-06 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695793+0000", 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3000000000000003e-06 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710881+0000", 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015087695999999999 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.359 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710947+0000", 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6169999999999998e-05 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.711013+0000", 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5643999999999997e-05 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.360 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95080 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625789220999998, 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014948011000000001, 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95080 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.362 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697385+0000", 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.187e-06 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697388+0000", 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9459999999999998e-06 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697390+0000", 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5600000000000001e-06 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.363 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712211+0000", 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014820224 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712276+0000", 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5650999999999999e-05 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712331+0000", 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4727e-05 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.364 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95081 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625779764999997, 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015124461000000001, 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95081 2024-09-13T07:14:05.365 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:05.366 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697394+0000", 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.418e-06 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697397+0000", 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.367 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697399+0000", 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.508e-06 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712367+0000", 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01496809 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712442+0000", 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5290999999999995e-05 2024-09-13T07:14:05.368 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712517+0000", 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4396999999999999e-05 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95082 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625771081000003, 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015249593000000001, 2024-09-13T07:14:05.369 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95082 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.370 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697402+0000", 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.311e-06 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.371 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697405+0000", 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697407+0000", 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712557+0000", 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015149207 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.372 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712609+0000", 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2286e-05 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712651+0000", 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1687999999999998e-05 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.373 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95083 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3727360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625762590999997, 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015418032, 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95083 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:05.374 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697409+0000", 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.375 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.049e-06 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697414+0000", 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.783e-06 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697417+0000", 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.587e-06 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.376 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712683+0000", 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015266083 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712759+0000", 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6627999999999999e-05 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712828+0000", 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8121999999999996e-05 2024-09-13T07:14:05.377 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95084 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3776512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625753320000001, 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015560297000000001, 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95084 2024-09-13T07:14:05.378 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.379 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697420+0000", 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697423+0000", 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3359999999999999e-06 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.380 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697425+0000", 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.452e-06 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712864+0000", 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015438927999999999 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712938+0000", 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3458e-05 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.381 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712979+0000", 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1251999999999997e-05 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95085 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625742529999997, 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015726161999999998, 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.382 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95085 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.383 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697429+0000", 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697431+0000", 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5230000000000001e-06 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.384 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697434+0000", 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697436+0000", 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3889999999999999e-06 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713008+0000", 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015571295000000001 2024-09-13T07:14:05.385 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713084+0000", 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6537000000000003e-05 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713156+0000", 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1616e-05 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.386 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95086 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3874816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625733525000001, 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015894697999999999, 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95086 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.387 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697438+0000", 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.388 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697440+0000", 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.159e-06 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697443+0000", 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9359999999999999e-06 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697445+0000", 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.389 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713191+0000", 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015746343999999999 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713269+0000", 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7737000000000005e-05 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713333+0000", 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4233000000000007e-05 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.390 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95087 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625725185999997, 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01604939, 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95087 2024-09-13T07:14:05.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.392 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697448+0000", 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.094e-06 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697451+0000", 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8949999999999998e-06 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.393 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697454+0000", 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.999e-06 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713361+0000", 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015907092000000001 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713433+0000", 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2606000000000002e-05 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.394 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713496+0000", 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2908000000000001e-05 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95088 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625715046000003, 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016219289000000001, 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.395 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95088 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:05.396 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697459+0000", 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.764e-06 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.397 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697462+0000", 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4240000000000002e-06 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697464+0000", 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3989999999999998e-06 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713532+0000", 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016067403000000001 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.398 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713607+0000", 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5476999999999996e-05 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713676+0000", 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9115999999999994e-05 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95089 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:05.399 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625705193000002, 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016381144, 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95089 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:05.400 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.401 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697468+0000", 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.734e-06 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697474+0000", 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697479+0000", 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.402 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713703+0000", 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016224643 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4828999999999998e-05 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713848+0000", 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0029000000000003e-05 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95090 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625689395000002, 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016573104000000002, 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95090 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.404 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697482+0000", 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697484+0000", 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.916e-06 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697496+0000", 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697506+0000", 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0477e-05 2024-09-13T07:14:05.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713903+0000", 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016396758000000001 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713981+0000", 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7892999999999994e-05 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714056+0000", 2024-09-13T07:14:05.407 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5214000000000002e-05 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95091 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625655973999997, 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016674768, 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.408 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95091 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.409 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697518+0000", 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4789999999999999e-06 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697527+0000", 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3799999999999994e-06 2024-09-13T07:14:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697532+0000", 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714080+0000", 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016547798999999998 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714135+0000", 2024-09-13T07:14:05.411 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5294000000000003e-05 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714191+0000", 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5606999999999998e-05 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95092 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4169728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:14:05.412 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 37.625635185999997, 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017373829, 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95092 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697536+0000", 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697540+0000", 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4879999999999999e-06 2024-09-13T07:14:05.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697548+0000", 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0139999999999998e-06 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697553+0000", 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2460000000000003e-06 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714754+0000", 2024-09-13T07:14:05.415 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017200653 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714831+0000", 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7156000000000005e-05 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714911+0000", 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9740999999999994e-05 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.416 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.417 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.417 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:05.418 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:14:05.418 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:05.419+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:05.518 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:05.519+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:05.561 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:05.562 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:05.562 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:05.562 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:05.570 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:14:05.619 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:14:05.620+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:14:05.714 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:05.714 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:05.714 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:05.714 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:05.714 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:05.719 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:05.720+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:05.722 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:14:05.821 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:05.822+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:14:05.847 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 52.472190148999999, 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99762887700000002, 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:14:05.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351641+0000", 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99754909000000003 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351653+0000", 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2248e-05 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88604 3.e 3:75ae40da:::rbd_header.117390733e87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.103359056000002, 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20280552800000001, 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.852 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88604 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.853 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722855+0000", 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722859+0000", 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722870+0000", 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0434999999999999e-05 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722875+0000", 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722912+0000", 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6943000000000002e-05 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722963+0000", 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1103000000000002e-05 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723070+0000", 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106279 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723472+0000", 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040248200000000001 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925590+0000", 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20211748900000001 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925624+0000", 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4801000000000001e-05 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925661+0000", 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6643999999999997e-05 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.857 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88605 3.e 3:75ae40da:::rbd_header.117390733e87:head [watch ping cookie 139824472318400 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 42.053151989, 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15274379900000001, 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.858 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88605 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773062+0000", 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773066+0000", 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1570000000000002e-06 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773073+0000", 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5490000000000003e-06 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773080+0000", 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773125+0000", 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4849999999999999e-05 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for rw locks", 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773190+0000", 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5567000000000004e-05 2024-09-13T07:14:05.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925689+0000", 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15249922899999999 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925746+0000", 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6369999999999997e-05 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925806+0000", 2024-09-13T07:14:05.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0668999999999998e-05 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94703 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3366912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.141033939000003, 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027375494, 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94703 2024-09-13T07:14:05.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685180+0000", 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685183+0000", 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685189+0000", 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0730000000000001e-06 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685193+0000", 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712474+0000", 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027280787000000001 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712507+0000", 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2786000000000001e-05 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712556+0000", 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9203000000000003e-05 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94704 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.141016551, 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027564668000000001, 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94704 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685200+0000", 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685207+0000", 2024-09-13T07:14:05.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685212+0000", 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1449999999999999e-06 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712597+0000", 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027385158 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712681+0000", 2024-09-13T07:14:05.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3961000000000005e-05 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712763+0000", 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1558999999999996e-05 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94705 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140986525999999, 2024-09-13T07:14:05.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027678611999999998, 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94705 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685230+0000", 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4660000000000002e-06 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685237+0000", 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685242+0000", 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8729999999999998e-06 2024-09-13T07:14:05.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712793+0000", 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027551022000000001 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712835+0000", 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2537000000000003e-05 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712907+0000", 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1556000000000004e-05 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94706 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140968985000001, 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027861107999999999, 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94706 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685245+0000", 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685247+0000", 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9479999999999998e-06 2024-09-13T07:14:05.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685253+0000", 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685258+0000", 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.549e-06 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712946+0000", 2024-09-13T07:14:05.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027688252 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713033+0000", 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7472000000000005e-05 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713107+0000", 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3478000000000004e-05 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94707 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140953455000002, 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028004273, 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94707 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685263+0000", 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8449999999999999e-06 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685283+0000", 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0165000000000001e-05 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685289+0000", 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7189999999999998e-06 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713138+0000", 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027849056000000001 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713189+0000", 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1801000000000001e-05 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713265+0000", 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5983999999999996e-05 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.884 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94708 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140921116999998, 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028161510000000001, 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94708 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685296+0000", 2024-09-13T07:14:05.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-06 2024-09-13T07:14:05.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685302+0000", 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685306+0000", 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713301+0000", 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027995176 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713372+0000", 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0870000000000004e-05 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713455+0000", 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2956999999999999e-05 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.889 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94709 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140904278000001, 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357679, 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94709 2024-09-13T07:14:05.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:05.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685312+0000", 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685318+0000", 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685323+0000", 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713505+0000", 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028182713000000002 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713592+0000", 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6397000000000006e-05 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.893 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713668+0000", 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6188999999999998e-05 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94710 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140886821999999, 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028518801, 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.894 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94710 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.895 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685327+0000", 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685330+0000", 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3120000000000001e-06 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.896 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685337+0000", 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685342+0000", 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713700+0000", 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357838999999999 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.897 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8438999999999999e-05 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713847+0000", 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8540000000000004e-05 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94711 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.898 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140868148999999, 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640149, 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94711 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:05.899 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.900 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685348+0000", 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685355+0000", 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685359+0000", 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.901 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713875+0000", 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028516414 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713931+0000", 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713987+0000", 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6084e-05 2024-09-13T07:14:05.902 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94712 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140851261000002, 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028779301, 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.903 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94712 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.904 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685365+0000", 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685371+0000", 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7620000000000001e-06 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.905 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685375+0000", 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.42e-06 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714016+0000", 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640591 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714079+0000", 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3288999999999997e-05 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.906 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714143+0000", 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3361999999999994e-05 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94713 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140835048, 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028911175000000001, 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.907 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94713 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.908 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685379+0000", 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685381+0000", 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1509999999999998e-06 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685387+0000", 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:14:05.909 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685391+0000", 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5920000000000002e-06 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714169+0000", 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028777971999999999 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714235+0000", 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5782000000000001e-05 2024-09-13T07:14:05.910 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714291+0000", 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5525000000000001e-05 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94714 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140794769999999, 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029006134999999999, 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.911 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94714 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:14:05.912 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685419+0000", 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685430+0000", 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015e-05 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.913 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685441+0000", 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1049e-05 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685446+0000", 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714313+0000", 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866640999999998 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.914 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714368+0000", 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5024e-05 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7843000000000002e-05 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94715 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140763073000002, 2024-09-13T07:14:05.915 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029140899000000001, 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94715 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.916 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685451+0000", 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685454+0000", 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.627e-06 2024-09-13T07:14:05.917 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685466+0000", 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2510000000000001e-05 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685472+0000", 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714453+0000", 2024-09-13T07:14:05.918 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028980599999999999 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714524+0000", 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1186999999999995e-05 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714592+0000", 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8668000000000001e-05 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94716 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140738161000002, 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029223545, 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94716 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.920 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.921 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685479+0000", 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4980000000000001e-06 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685485+0000", 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.922 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685489+0000", 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714617+0000", 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029128106000000001 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714662+0000", 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.456e-05 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.923 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714700+0000", 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8281000000000002e-05 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94717 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140721255999999, 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029398906999999998, 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.924 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94717 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.925 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685495+0000", 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0499999999999999e-06 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685501+0000", 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9569999999999999e-06 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.926 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685506+0000", 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3249999999999998e-06 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714726+0000", 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029219218000000002 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714804+0000", 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8610999999999997e-05 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.927 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714892+0000", 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8114000000000004e-05 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94718 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140703977000001, 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029526177000000001, 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.928 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94718 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.929 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685510+0000", 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685513+0000", 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4360000000000001e-06 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685519+0000", 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.930 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685523+0000", 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714918+0000", 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029395260999999999 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714985+0000", 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6737999999999997e-05 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.931 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715037+0000", 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1712999999999998e-05 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94719 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 38.140687460000002, 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029657744, 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:05.932 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94719 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:05.933 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685529+0000", 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685535+0000", 2024-09-13T07:14:05.934 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685539+0000", 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715062+0000", 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029522738 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715125+0000", 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3229000000000001e-05 2024-09-13T07:14:05.935 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715185+0000", 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9441999999999998e-05 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:05.936 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:05.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:14:05.938 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:05.939+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:05.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:14:06.039 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:06.040+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:06.058 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:06.058 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T07:14:06.058 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T07:14:06.058 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:06.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:14:06.113 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:14:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:14:06.130 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:14:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:14:06.140 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:06.141+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:06.148 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:14:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:14:06.165 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:14:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:14:06.179 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:06.179 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:06.182 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:14:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:14:06.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:14:06.199 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:14:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:14:06.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:14:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:14:06.233 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:14:06.233 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:14:06.241 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:06.242+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:06.305 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:06.305 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:06.316 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:14:06.360 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:14:06.361+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:14:06.426 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:06.426 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:06.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:14:06.443 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:06.444+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:06.544 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:06.545+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:06.582 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:06.582 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:06.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:14:06.674 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:06.675+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:06.737 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:06.737 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:06.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:14:06.786 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T07:14:06.787+0000 7fd3935cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33283) UID: 0 2024-09-13T07:14:06.848 INFO:tasks.ceph.osd.6.smithi184.stderr:2024-09-13T07:14:06.849+0000 7fa9e9182640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33025) UID: 0 2024-09-13T07:14:06.892 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:06.892 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:06.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:14:06.949 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T07:14:06.950+0000 7f08fee81640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33295) UID: 0 2024-09-13T07:14:07.007 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.007 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:14:07.057 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T07:14:07.058+0000 7f0ac7334640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33284) UID: 0 2024-09-13T07:14:07.151 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:07.152+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:14:07.252 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:07.253+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:07.280 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.280 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:14:07.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:14:07.375 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:07.376+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:07.454 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:07.454+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:07.455 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.455 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:14:07.554 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:07.555+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:07.613 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.613 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.621 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:14:07.655 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:07.656+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:07.716 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:14:07.717 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:14:07.717 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:14:07.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:14:07.727 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.727 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.736 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:14:07.761 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:07.762+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:07.838 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.838 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.847 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:14:07.857 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:07.857+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:07.949 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:07.949 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:07.958 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:14:07.958 INFO:tasks.ceph.osd.7.smithi184.stderr:2024-09-13T07:14:07.959+0000 7efee0639640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 41938) UID: 0 2024-09-13T07:14:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:08.001 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 105, 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "f667c631-bb9d-47bf-97c6-7217d8893244", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T07:12:08.346514+0000", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T07:14:07.270363+0000", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T07:13:13.343075+0000", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T07:12:10.626165+0000", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T07:14:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T07:14:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:19.268005+0000", 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:14:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:14:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T07:14:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 17, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:14:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:14:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T07:14:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:20.513150+0000", 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:14:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:14:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T07:14:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:14:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:14:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T07:14:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T07:12:28.796583+0000", 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T07:14:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T07:14:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "105", 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 72, 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 105, 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T07:14:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T07:14:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T07:14:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.98191452026367188, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c29fb342-d45b-48c1-9694-dd01e2a01325", 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 66, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:14:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1709347595 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/1709347595", 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/1709347595", 2024-09-13T07:14:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/1709347595", 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/1709347595", 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4b979b5c-a78b-461d-b795-f6f5c25cf652", 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.855316162109375, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:14:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 65, 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T07:14:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T07:14:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3300827638 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3300827638", 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3300827638", 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3300827638", 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3300827638", 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "11259f54-1134-495f-9281-d5132aff2780", 2024-09-13T07:14:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T07:14:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3977233096 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3977233096", 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3977233096", 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3977233096", 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3977233096", 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "a11be2be-3803-4a23-bbf8-40ea110ccf41", 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:14:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T07:14:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3290545828 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/3290545828", 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/3290545828", 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/3290545828", 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/3290545828", 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2b6941d9-29f5-4836-a61c-47ff27a7172e", 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6800", 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6801", 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6802", 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6803", 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6806", 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6807", 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6804", 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6805", 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3543694628 2024-09-13T07:14:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6801/3543694628", 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6803/3543694628", 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6807/3543694628", 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6805/3543694628", 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "17e3a02d-28e3-4126-a8f2-c7783e524d62", 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 55, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6808", 2024-09-13T07:14:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6810", 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6812", 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6814", 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6820", 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6821", 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6816", 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6818", 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 951246042 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6810/951246042", 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6814/951246042", 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6821/951246042", 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6818/951246042", 2024-09-13T07:14:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ddf29f57-7733-42fa-94ed-015a529fb689", 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T07:14:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6824", 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6825", 2024-09-13T07:14:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6826", 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6827", 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6830", 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6831", 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6828", 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6829", 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1941674146 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6825/1941674146", 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6827/1941674146", 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6831/1941674146", 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6829/1941674146", 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d91e2eed-3bfb-4eb0-ac30-18770c106742", 2024-09-13T07:14:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 13, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 30, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 53, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 31, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6809", 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6811", 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6813", 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6815", 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6822", 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6823", 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T07:14:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6817", 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.184:6819", 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 48890479 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.184:6811/48890479", 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.184:6815/48890479", 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.184:6823/48890479", 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.184:6819/48890479", 2024-09-13T07:14:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.805494+0000", 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.817444+0000", 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.712083+0000", 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.739958+0000", 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.720767+0000", 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.715944+0000", 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.788853+0000", 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T07:12:15.809325+0000", 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 31 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-13T07:14:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.8", 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: 0, 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T07:14:08.071 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 5, 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 10 2024-09-13T07:14:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 5, 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T07:14:08.074 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:08.076 INFO:tasks.thrashosds.thrasher:j is {'epoch': 105, 'fsid': 'f667c631-bb9d-47bf-97c6-7217d8893244', 'created': '2024-09-13T07:12:08.346514+0000', 'modified': '2024-09-13T07:14:07.270363+0000', 'last_up_change': '2024-09-13T07:13:13.343075+0000', 'last_in_change': '2024-09-13T07:12:10.626165+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:12:19.268005+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:12:20.513150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:12:28.796583+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 72, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9819145202636719, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'c29fb342-d45b-48c1-9694-dd01e2a01325', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 1709347595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 1709347595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 1709347595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 1709347595}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 1709347595}]}, 'public_addr': '172.21.15.31:6809/1709347595', 'cluster_addr': '172.21.15.31:6811/1709347595', 'heartbeat_back_addr': '172.21.15.31:6815/1709347595', 'heartbeat_front_addr': '172.21.15.31:6813/1709347595', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4b979b5c-a78b-461d-b795-f6f5c25cf652', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.855316162109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3300827638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3300827638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3300827638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3300827638}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3300827638}]}, 'public_addr': '172.21.15.31:6825/3300827638', 'cluster_addr': '172.21.15.31:6827/3300827638', 'heartbeat_back_addr': '172.21.15.31:6831/3300827638', 'heartbeat_front_addr': '172.21.15.31:6829/3300827638', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '11259f54-1134-495f-9281-d5132aff2780', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3977233096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3977233096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3977233096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3977233096}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3977233096}]}, 'public_addr': '172.21.15.31:6801/3977233096', 'cluster_addr': '172.21.15.31:6803/3977233096', 'heartbeat_back_addr': '172.21.15.31:6807/3977233096', 'heartbeat_front_addr': '172.21.15.31:6805/3977233096', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a11be2be-3803-4a23-bbf8-40ea110ccf41', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 3290545828}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 3290545828}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 3290545828}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3290545828}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 3290545828}]}, 'public_addr': '172.21.15.31:6817/3290545828', 'cluster_addr': '172.21.15.31:6819/3290545828', 'heartbeat_back_addr': '172.21.15.31:6823/3290545828', 'heartbeat_front_addr': '172.21.15.31:6821/3290545828', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2b6941d9-29f5-4836-a61c-47ff27a7172e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6800', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6801', 'nonce': 3543694628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6802', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6803', 'nonce': 3543694628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6806', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6807', 'nonce': 3543694628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6804', 'nonce': 3543694628}, {'type': 'v1', 'addr': '172.21.15.184:6805', 'nonce': 3543694628}]}, 'public_addr': '172.21.15.184:6801/3543694628', 'cluster_addr': '172.21.15.184:6803/3543694628', 'heartbeat_back_addr': '172.21.15.184:6807/3543694628', 'heartbeat_front_addr': '172.21.15.184:6805/3543694628', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '17e3a02d-28e3-4126-a8f2-c7783e524d62', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6808', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6810', 'nonce': 951246042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6812', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6814', 'nonce': 951246042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6820', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6821', 'nonce': 951246042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6816', 'nonce': 951246042}, {'type': 'v1', 'addr': '172.21.15.184:6818', 'nonce': 951246042}]}, 'public_addr': '172.21.15.184:6810/951246042', 'cluster_addr': '172.21.15.184:6814/951246042', 'heartbeat_back_addr': '172.21.15.184:6821/951246042', 'heartbeat_front_addr': '172.21.15.184:6818/951246042', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ddf29f57-7733-42fa-94ed-015a529fb689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6824', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6825', 'nonce': 1941674146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6826', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6827', 'nonce': 1941674146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6830', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6831', 'nonce': 1941674146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6828', 'nonce': 1941674146}, {'type': 'v1', 'addr': '172.21.15.184:6829', 'nonce': 1941674146}]}, 'public_addr': '172.21.15.184:6825/1941674146', 'cluster_addr': '172.21.15.184:6827/1941674146', 'heartbeat_back_addr': '172.21.15.184:6831/1941674146', 'heartbeat_front_addr': '172.21.15.184:6829/1941674146', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd91e2eed-3bfb-4eb0-ac30-18770c106742', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 30, 'up_from': 53, 'up_thru': 56, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6809', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6811', 'nonce': 48890479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6813', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6815', 'nonce': 48890479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6822', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6823', 'nonce': 48890479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6817', 'nonce': 48890479}, {'type': 'v1', 'addr': '172.21.15.184:6819', 'nonce': 48890479}]}, 'public_addr': '172.21.15.184:6811/48890479', 'cluster_addr': '172.21.15.184:6815/48890479', 'heartbeat_back_addr': '172.21.15.184:6823/48890479', 'heartbeat_front_addr': '172.21.15.184:6819/48890479', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.805494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.817444+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.712083+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.739958+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.720767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.715944+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.788853+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:12:15.809325+0000', 'dead_epoch': 31}], 'pg_upmap': [{'pgid': '3.8', 'osds': [0, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 5, 'length': 10}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 5, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:14:08.076 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.8 2024-09-13T07:14:08.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.8 2024-09-13T07:14:08.077 INFO:tasks.ceph.osd.5.smithi184.stderr:2024-09-13T07:14:08.076+0000 7f884a98f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33022) UID: 0 2024-09-13T07:14:08.100 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:08.100 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:08.108 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:14:08.146 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T07:14:08.202 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T07:14:08.203+0000 7f070307f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33294) UID: 0 2024-09-13T07:14:08.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:14:08.260 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:08.261 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:08.271 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:14:08.283 INFO:teuthology.orchestra.run.smithi031.stdout:All operations completed A-OK! 2024-09-13T07:14:08.297 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:08.297 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T07:14:08.306 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T07:14:08.429 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T07:14:08.430 INFO:tasks.ceph.osd.4.smithi184.stderr:2024-09-13T07:14:08.315+0000 7f35da2d9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33019) UID: 0 2024-09-13T07:14:08.431 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T07:14:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:14:08.431 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T07:14:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:14:08.432 INFO:teuthology.orchestra.run.smithi184.stdout:ERROR: (22) Invalid argument 2024-09-13T07:14:08.432 INFO:teuthology.orchestra.run.smithi184.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:14:08.432 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:14:08.433 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T07:14:08.433 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:14:08.433 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T07:14:08.434 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:14:08.437 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T07:14:08.437 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:14:08.454 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T07:14:08.455 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:14:08.471 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T07:14:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:14:08.487 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T07:14:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:14:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:08.574 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:08.583 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:14:08.685 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:08.685 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:08.685 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:08.685 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:08.685 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:08.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:14:08.798 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:08.798 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:08.798 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:08.798 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:08.798 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.798 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:1768 3.a 3:53b8956c:::rbd_data.1173c53667c.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 90.761002536999996, 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.024994197999999999, 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 1768 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.799 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037192+0000", 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037191+0000", 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.800 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037200+0000", 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4720000000000001e-06 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037217+0000", 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037223+0000", 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.801 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037269+0000", 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5649000000000003e-05 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.037364+0000", 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.5117000000000004e-05 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.042436+0000", 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0050717389999999996 2024-09-13T07:14:08.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062017+0000", 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.019581173 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062067+0000", 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9849000000000003e-05 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:08.803 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062119+0000", 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1863999999999997e-05 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:12:38.062186+0000", 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.7692000000000002e-05 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:71808 3.a 3:552625c6:::rbd_data.1173ab534c79.0000000000000006:head [stat,truncate 2490368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:14:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 59.214093665, 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.042237893999999998, 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 71808 2024-09-13T07:14:08.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.806 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.806 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.806 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.806 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:14:08.806 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584101+0000", 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584099+0000", 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999981 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584107+0000", 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8169999999999999e-06 2024-09-13T07:14:08.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584128+0000", 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0682e-05 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584136+0000", 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9880000000000004e-06 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584172+0000", 2024-09-13T07:14:08.808 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6872000000000002e-05 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584222+0000", 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9440000000000001e-05 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584295+0000", 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3058000000000004e-05 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.809 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.584842+0000", 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00054750399999999998 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626236+0000", 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.041393359999999997 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626298+0000", 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.1760999999999994e-05 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.810 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:09.626339+0000", 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.155e-05 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88519 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.212935831000003, 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017910447999999999, 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:08.811 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88519 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585259+0000", 2024-09-13T07:14:08.812 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585257+0000", 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999976 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585267+0000", 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0635000000000001e-05 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.813 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585289+0000", 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1846999999999999e-05 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585299+0000", 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 9.4949999999999997e-06 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585347+0000", 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7672e-05 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.814 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.585500+0000", 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00015363700000000001 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.588596+0000", 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0030953669999999999 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603068+0000", 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014472063 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:08.815 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603106+0000", 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8435e-05 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603142+0000", 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6488e-05 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.603169+0000", 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.6959000000000001e-05 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.816 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:89021 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 44.268458918, 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.02102269, 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 89021 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.817 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529736+0000", 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529734+0000", 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999986 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.818 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529742+0000", 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8290000000000008e-06 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529757+0000", 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.4868999999999999e-05 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529763+0000", 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.93e-06 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.819 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529803+0000", 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0404999999999997e-05 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op must wait for map", 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.529809+0000", 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2059999999999998e-06 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531015+0000", 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0012062049999999999 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.820 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.531057+0000", 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1921000000000001e-05 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 0", 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.534526+0000", 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.0034694909999999999 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550683+0000", 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016157169999999998 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.821 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550697+0000", 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.3351e-05 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550731+0000", 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3997e-05 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:24.550759+0000", 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.7861e-05 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.822 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95026 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.102444851999998, 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014972104999999999, 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95026 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.823 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695750+0000", 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695752+0000", 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.283e-06 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695758+0000", 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.825 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695762+0000", 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710581+0000", 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014818934000000001 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710653+0000", 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2457000000000001e-05 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.826 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710722+0000", 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8633999999999994e-05 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95027 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.102428742999997, 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015079646, 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.827 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95027 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695766+0000", 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695768+0000", 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695773+0000", 2024-09-13T07:14:08.829 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.592e-06 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695778+0000", 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710749+0000", 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014970809 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.830 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710805+0000", 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6149000000000001e-05 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710846+0000", 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.0840999999999997e-05 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95028 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.831 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.102413046999999, 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015231216000000001, 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95028 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:14:08.832 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695782+0000", 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695781+0000", 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.833 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695783+0000", 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9250000000000002e-06 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695789+0000", 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.835e-06 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.695793+0000", 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.3000000000000003e-06 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.834 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710881+0000", 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015087695999999999 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.710947+0000", 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6169999999999998e-05 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.711013+0000", 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5643999999999997e-05 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.835 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95080 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100811919999998, 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014948011000000001, 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95080 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.836 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697383+0000", 2024-09-13T07:14:08.837 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697385+0000", 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.187e-06 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697388+0000", 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9459999999999998e-06 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697390+0000", 2024-09-13T07:14:08.838 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5600000000000001e-06 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712211+0000", 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.014820224 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712276+0000", 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5650999999999999e-05 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.839 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712331+0000", 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4727e-05 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95081 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100802463999997, 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015124461000000001, 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.840 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95081 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697392+0000", 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697394+0000", 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.418e-06 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697397+0000", 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:14:08.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697399+0000", 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.508e-06 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712367+0000", 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01496809 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.843 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712442+0000", 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5290999999999995e-05 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712517+0000", 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4396999999999999e-05 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95082 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:08.844 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100793779999997, 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015249593000000001, 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95082 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697401+0000", 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697402+0000", 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.311e-06 2024-09-13T07:14:08.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697405+0000", 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697407+0000", 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4609999999999998e-06 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712557+0000", 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015149207 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.847 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712609+0000", 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2286e-05 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712651+0000", 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1687999999999998e-05 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95083 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3727360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.848 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100785289999997, 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015418032, 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95083 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697409+0000", 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697410+0000", 2024-09-13T07:14:08.850 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.049e-06 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697414+0000", 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.783e-06 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697417+0000", 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.587e-06 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712683+0000", 2024-09-13T07:14:08.851 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015266083 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712759+0000", 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6627999999999999e-05 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712828+0000", 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8121999999999996e-05 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.852 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95084 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3776512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100776019000001, 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015560297000000001, 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95084 2024-09-13T07:14:08.853 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.854 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697419+0000", 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697420+0000", 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0920000000000001e-06 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697423+0000", 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.3359999999999999e-06 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.855 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697425+0000", 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.452e-06 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712864+0000", 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015438927999999999 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712938+0000", 2024-09-13T07:14:08.856 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3458e-05 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712979+0000", 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1251999999999997e-05 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95085 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3825664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100765228999997, 2024-09-13T07:14:08.857 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015726161999999998, 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95085 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697430+0000", 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697429+0000", 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697431+0000", 2024-09-13T07:14:08.859 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.5230000000000001e-06 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697434+0000", 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.0580000000000002e-06 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697436+0000", 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3889999999999999e-06 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.860 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713008+0000", 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015571295000000001 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713084+0000", 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6537000000000003e-05 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713156+0000", 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1616e-05 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.861 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95086 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3874816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100756224000001, 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015894697999999999, 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95086 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.862 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697439+0000", 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697438+0000", 2024-09-13T07:14:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697440+0000", 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.159e-06 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697443+0000", 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.9359999999999999e-06 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697445+0000", 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713191+0000", 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015746343999999999 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713269+0000", 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7737000000000005e-05 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.865 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713333+0000", 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.4233000000000007e-05 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95087 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100747884999997, 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.01604939, 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.866 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95087 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.867 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697447+0000", 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967296 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697448+0000", 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.094e-06 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.868 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697451+0000", 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.8949999999999998e-06 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697454+0000", 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.999e-06 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713361+0000", 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.015907092000000001 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.869 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713433+0000", 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.2606000000000002e-05 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713496+0000", 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.2908000000000001e-05 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.870 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95088 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100737745000004, 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016219289000000001, 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95088 2024-09-13T07:14:08.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.872 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697457+0000", 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697459+0000", 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.764e-06 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697462+0000", 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4240000000000002e-06 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.873 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697464+0000", 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3989999999999998e-06 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713532+0000", 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016067403000000001 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713607+0000", 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5476999999999996e-05 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.874 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713676+0000", 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9115999999999994e-05 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95089 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100727892000002, 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016381144, 2024-09-13T07:14:08.875 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95089 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:08.876 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697467+0000", 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697468+0000", 2024-09-13T07:14:08.877 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.734e-06 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697474+0000", 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697479+0000", 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9200000000000003e-06 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.878 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713703+0000", 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016224643 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.4828999999999998e-05 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713848+0000", 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0029000000000003e-05 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95090 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100712094000002, 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016573104000000002, 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95090 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.880 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697483+0000", 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.881 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697482+0000", 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697484+0000", 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.916e-06 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697496+0000", 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.882 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697506+0000", 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0477e-05 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713903+0000", 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016396758000000001 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713981+0000", 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7892999999999994e-05 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.883 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714056+0000", 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5214000000000002e-05 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95091 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100678672999997, 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016674768, 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.884 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95091 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.885 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697516+0000", 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697518+0000", 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4789999999999999e-06 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.886 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697527+0000", 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3799999999999994e-06 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697532+0000", 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6110000000000001e-06 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714080+0000", 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.016547798999999998 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714135+0000", 2024-09-13T07:14:08.887 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5294000000000003e-05 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714191+0000", 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5606999999999998e-05 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:95092 3.10 3:0fbde0a3:::rbd_data.117390733e87.000000000000000a:21 [read 4169728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.100657884999997, 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017373829, 2024-09-13T07:14:08.888 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 95092 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:08.889 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697537+0000", 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697536+0000", 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697540+0000", 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4879999999999999e-06 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.890 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697548+0000", 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.0139999999999998e-06 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.697553+0000", 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.2460000000000003e-06 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714754+0000", 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.017200653 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.891 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714831+0000", 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7156000000000005e-05 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714911+0000", 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.9740999999999994e-05 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:08.892 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:08.894 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:14:08.996 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:08.996 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:08.996 INFO:teuthology.orchestra.run.smithi184.stdout: "num_ops": 0 2024-09-13T07:14:08.996 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:09.005 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:14:09.107 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:09.108 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [], 2024-09-13T07:14:09.108 INFO:teuthology.orchestra.run.smithi184.stdout: "complaint_time": 30, 2024-09-13T07:14:09.108 INFO:teuthology.orchestra.run.smithi184.stdout: "num_blocked_ops": 0 2024-09-13T07:14:09.108 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:09.116 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "size": 20, 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 600, 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "ops": [ 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:78853 3.f 3.d55f56af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 55.866844036000003, 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99762887700000002, 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.221 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "reached pg", 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 78853 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354024+0000", 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.222 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354023+0000", 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354029+0000", 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4099999999999999e-06 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354037+0000", 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.7999999999999999e-06 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.223 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354043+0000", 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354085+0000", 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2487000000000002e-05 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for peered", 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:13.354092+0000", 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.224 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351641+0000", 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.99754909000000003 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:14.351653+0000", 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2248e-05 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.225 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88604 3.e 3:75ae40da:::rbd_header.117390733e87:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.498012942999999, 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20280552800000001, 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88604 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.226 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722856+0000", 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.227 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722855+0000", 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.999999 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722859+0000", 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7809999999999996e-06 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722870+0000", 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.0434999999999999e-05 2024-09-13T07:14:09.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722875+0000", 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722912+0000", 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6943000000000002e-05 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.229 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.722963+0000", 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1103000000000002e-05 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for subops from 4", 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723070+0000", 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.000106279 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "op_commit", 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.723472+0000", 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.00040248200000000001 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.230 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925590+0000", 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.20211748900000001 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "commit_sent", 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925624+0000", 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.4801000000000001e-05 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925661+0000", 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.6643999999999997e-05 2024-09-13T07:14:09.231 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.232 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.232 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.232 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.232 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.232 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:88605 3.e 3:75ae40da:::rbd_header.117390733e87:head [watch ping cookie 139824472318400 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T07:14:09.232 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:14:09.232 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 45.447805875999997, 2024-09-13T07:14:09.233 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15274379900000001, 2024-09-13T07:14:09.233 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.233 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.233 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.233 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 88605 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773063+0000", 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.234 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773062+0000", 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773066+0000", 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.1570000000000002e-06 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773073+0000", 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5490000000000003e-06 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.235 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773080+0000", 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9750000000000001e-06 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773125+0000", 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4849999999999999e-05 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "waiting for rw locks", 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.773190+0000", 2024-09-13T07:14:09.236 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5567000000000004e-05 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925689+0000", 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.15249922899999999 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925746+0000", 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6369999999999997e-05 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.237 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:23.925806+0000", 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0668999999999998e-05 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94703 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3366912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535687826, 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027375494, 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.238 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94703 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685181+0000", 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.239 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685180+0000", 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685183+0000", 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.240 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685189+0000", 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.0730000000000001e-06 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685193+0000", 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712474+0000", 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027280787000000001 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.241 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712507+0000", 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.2786000000000001e-05 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712556+0000", 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9203000000000003e-05 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94704 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3416064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.242 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535670437999997, 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027564668000000001, 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94704 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685198+0000", 2024-09-13T07:14:09.244 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685200+0000", 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685207+0000", 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.245 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685212+0000", 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1449999999999999e-06 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712597+0000", 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027385158 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712681+0000", 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.3961000000000005e-05 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.246 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712763+0000", 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.1558999999999996e-05 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94705 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535640413000003, 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027678611999999998, 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.247 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.248 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.248 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94705 2024-09-13T07:14:09.248 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.248 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.248 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.248 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.248 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:09.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.249 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.249 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.249 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:09.249 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.249 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.250 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.250 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685228+0000", 2024-09-13T07:14:09.250 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.250 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685230+0000", 2024-09-13T07:14:09.251 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4660000000000002e-06 2024-09-13T07:14:09.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.251 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.251 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.251 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685237+0000", 2024-09-13T07:14:09.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5780000000000004e-06 2024-09-13T07:14:09.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.252 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.252 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.252 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685242+0000", 2024-09-13T07:14:09.252 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.8729999999999998e-06 2024-09-13T07:14:09.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.253 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.253 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.253 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712793+0000", 2024-09-13T07:14:09.253 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027551022000000001 2024-09-13T07:14:09.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.254 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.254 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712835+0000", 2024-09-13T07:14:09.254 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.2537000000000003e-05 2024-09-13T07:14:09.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.254 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.255 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.255 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712907+0000", 2024-09-13T07:14:09.255 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1556000000000004e-05 2024-09-13T07:14:09.255 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.255 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.255 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.256 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.256 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94706 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.256 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:14:09.256 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535622871999998, 2024-09-13T07:14:09.256 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027861107999999999, 2024-09-13T07:14:09.257 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.257 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.257 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.257 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94706 2024-09-13T07:14:09.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.258 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.258 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.258 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.258 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:14:09.258 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.259 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.259 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.259 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685246+0000", 2024-09-13T07:14:09.259 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685245+0000", 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685247+0000", 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.9479999999999998e-06 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.260 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685253+0000", 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8440000000000003e-06 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685258+0000", 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.549e-06 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.712946+0000", 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027688252 2024-09-13T07:14:09.261 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713033+0000", 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.7472000000000005e-05 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713107+0000", 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.3478000000000004e-05 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.262 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94707 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3563520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535607341999999, 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028004273, 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94707 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.263 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685261+0000", 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.264 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685263+0000", 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.8449999999999999e-06 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685283+0000", 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0165000000000001e-05 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685289+0000", 2024-09-13T07:14:09.265 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7189999999999998e-06 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713138+0000", 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027849056000000001 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713189+0000", 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1801000000000001e-05 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713265+0000", 2024-09-13T07:14:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.5983999999999996e-05 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94708 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3612672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535575004000002, 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028161510000000001, 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94708 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685293+0000", 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685296+0000", 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5780000000000001e-06 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685302+0000", 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:14:09.269 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685306+0000", 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713301+0000", 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.027995176 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713372+0000", 2024-09-13T07:14:09.270 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0870000000000004e-05 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713455+0000", 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.2956999999999999e-05 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94709 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:09.271 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535558164999998, 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357679, 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94709 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.272 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685310+0000", 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685312+0000", 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685318+0000", 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7400000000000001e-06 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.274 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685323+0000", 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5020000000000003e-06 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713505+0000", 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028182713000000002 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713592+0000", 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.6397000000000006e-05 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.275 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713668+0000", 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.6188999999999998e-05 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94710 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535540709000003, 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028518801, 2024-09-13T07:14:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94710 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.277 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685328+0000", 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685327+0000", 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685330+0000", 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.3120000000000001e-06 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.278 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685337+0000", 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.0400000000000004e-06 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685342+0000", 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9690000000000002e-06 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713700+0000", 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028357838999999999 2024-09-13T07:14:09.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713778+0000", 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8438999999999999e-05 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713847+0000", 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8540000000000004e-05 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94711 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535522036000003, 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640149, 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94711 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.281 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685346+0000", 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685348+0000", 2024-09-13T07:14:09.282 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1639999999999999e-06 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685355+0000", 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685359+0000", 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.476e-06 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.283 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713875+0000", 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028516414 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713931+0000", 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5089000000000001e-05 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.713987+0000", 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6084e-05 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.284 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94712 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535505147999999, 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028779301, 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94712 2024-09-13T07:14:09.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.286 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685363+0000", 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685365+0000", 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685371+0000", 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7620000000000001e-06 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.287 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685375+0000", 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.42e-06 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714016+0000", 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028640591 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714079+0000", 2024-09-13T07:14:09.288 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3288999999999997e-05 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714143+0000", 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3361999999999994e-05 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94713 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3858432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:14:09.289 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535488935000004, 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028911175000000001, 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94713 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.290 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685380+0000", 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685379+0000", 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685381+0000", 2024-09-13T07:14:09.291 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.1509999999999998e-06 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685387+0000", 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5890000000000002e-06 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685391+0000", 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5920000000000002e-06 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714169+0000", 2024-09-13T07:14:09.292 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028777971999999999 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714235+0000", 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.5782000000000001e-05 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714291+0000", 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5525000000000001e-05 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94714 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535448657000003, 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029006134999999999, 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94714 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.294 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685420+0000", 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685419+0000", 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685430+0000", 2024-09-13T07:14:09.295 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1015e-05 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685441+0000", 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.1049e-05 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685446+0000", 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.0579999999999997e-06 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714313+0000", 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028866640999999998 2024-09-13T07:14:09.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714368+0000", 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.5024e-05 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714426+0000", 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7843000000000002e-05 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.297 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94715 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535416959999999, 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029140899000000001, 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94715 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:14:09.298 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685452+0000", 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685451+0000", 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.299 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685454+0000", 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.627e-06 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685466+0000", 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 1.2510000000000001e-05 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685472+0000", 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:14:09.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714453+0000", 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.028980599999999999 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714524+0000", 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.1186999999999995e-05 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714592+0000", 2024-09-13T07:14:09.301 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.8668000000000001e-05 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94716 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535392047999999, 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029223545, 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.302 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94716 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.303 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685476+0000", 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685479+0000", 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4980000000000001e-06 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685485+0000", 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.037e-06 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.304 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685489+0000", 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714617+0000", 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029128106000000001 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714662+0000", 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.456e-05 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.305 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714700+0000", 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 3.8281000000000002e-05 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94717 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535375143000003, 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029398906999999998, 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.306 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94717 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.307 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685493+0000", 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685495+0000", 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0499999999999999e-06 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.308 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685501+0000", 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9569999999999999e-06 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685506+0000", 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.3249999999999998e-06 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714726+0000", 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029219218000000002 2024-09-13T07:14:09.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714804+0000", 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 7.8610999999999997e-05 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714892+0000", 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 8.8114000000000004e-05 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.310 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94718 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535357863999998, 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029526177000000001, 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94718 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.311 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685511+0000", 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685510+0000", 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.312 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685513+0000", 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.4360000000000001e-06 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685519+0000", 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685523+0000", 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:14:09.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714918+0000", 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029395260999999999 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.714985+0000", 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.6737999999999997e-05 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715037+0000", 2024-09-13T07:14:09.314 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.1712999999999998e-05 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "description": "osd_op(client.4467.0:94719 3.d 3:b7303a10:::rbd_data.117390733e87.0000000000000009:21 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "initiated_at": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "age": 41.535341346999999, 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029657744, 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "type_data": { 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "flag_point": "started", 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_info": { 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client": "client.4467", 2024-09-13T07:14:09.315 INFO:teuthology.orchestra.run.smithi184.stdout: "client_addr": "172.21.15.31:0/1562446403", 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "tid": 94719 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "events": [ 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "initiated", 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "throttled", 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.316 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "header_read", 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685527+0000", 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4294967295.9999995 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "all_read", 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685529+0000", 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 2.0679999999999999e-06 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "dispatched", 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685535+0000", 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.317 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "queued_for_pg", 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.685539+0000", 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "reached_pg", 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715062+0000", 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 0.029522738 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "started", 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715125+0000", 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 6.3229000000000001e-05 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: { 2024-09-13T07:14:09.318 INFO:teuthology.orchestra.run.smithi184.stdout: "event": "done", 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout: "time": "2024-09-13T07:13:27.715185+0000", 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout: "duration": 5.9441999999999998e-05 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-09-13T07:14:09.319 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-09-13T07:14:09.321 INFO:teuthology.orchestra.run.smithi031.stderr:clear 3.8 pg_upmap mapping 2024-09-13T07:14:10.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:14:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T07:14:11.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:14:12.306 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T07:14:14.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T07:14:14.451 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-13T07:14:14.452 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:14:14.461 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-13T07:14:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:14:14.470 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-13T07:14:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:14:14.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T07:14:15.321 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T07:14:15.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T07:14:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T07:14:16.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T07:14:17.348 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T07:14:17.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T07:14:18.151 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T07:14:18.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T07:14:19.348 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T07:14:19.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T07:14:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T07:14:20.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T07:14:21.365 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T07:14:21.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T07:14:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T07:14:22.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T07:14:23.385 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T07:14:23.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T07:14:24.394 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T07:14:24.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T07:14:25.402 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T07:14:25.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T07:14:26.409 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T07:14:26.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T07:14:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T07:14:27.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T07:14:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T07:14:28.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T07:14:29.431 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T07:14:29.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T07:14:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T07:14:30.455 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T07:14:30.455 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T07:14:30.455 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T07:14:30.455 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T07:14:30.456 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T07:14:30.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:14:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:30.748 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:14:30.434428+0000","last_up_change":"2024-09-13T07:13:13.343075+0000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T07:12:19.268005+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T07:12:20.513150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T07:12:28.796583+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":73,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6809","nonce":1709347595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6811","nonce":1709347595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6815","nonce":1709347595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6813","nonce":1709347595}]},"public_addr":"172.21.15.31:6809/1709347595","cluster_addr":"172.21.15.31:6811/1709347595","heartbeat_back_addr":"172.21.15.31:6815/1709347595","heartbeat_front_addr":"172.21.15.31:6813/1709347595","state":["exists","up"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6825","nonce":3300827638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6827","nonce":3300827638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6831","nonce":3300827638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6829","nonce":3300827638}]},"public_addr":"172.21.15.31:6825/3300827638","cluster_addr":"172.21.15.31:6827/3300827638","heartbeat_back_addr":"172.21.15.31:6831/3300827638","heartbeat_front_addr":"172.21.15.31:6829/3300827638","state":["exists","up"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6801","nonce":3977233096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6803","nonce":3977233096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6807","nonce":3977233096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6805","nonce":3977233096}]},"public_addr":"172.21.15.31:6801/3977233096","cluster_addr":"172.21.15.31:6803/3977233096","heartbeat_back_addr":"172.21.15.31:6807/3977233096","heartbeat_front_addr":"172.21.15.31:6805/3977233096","state":["exists","up"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6817","nonce":3290545828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6819","nonce":3290545828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6823","nonce":3290545828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6821","nonce":3290545828}]},"public_addr":"172.21.15.31:6817/3290545828","cluster_addr":"172.21.15.31:6819/3290545828","heartbeat_back_addr":"172.21.15.31:6823/3290545828","heartbeat_front_addr":"172.21.15.31:6821/3290545828","state":["exists","up"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6801","nonce":3543694628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6803","nonce":3543694628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6807","nonce":3543694628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6805","nonce":3543694628}]},"public_addr":"172.21.15.184:6801/3543694628","cluster_addr":"172.21.15.184:6803/3543694628","heartbeat_back_addr":"172.21.15.184:6807/3543694628","heartbeat_front_addr":"172.21.15.184:6805/3543694628","state":["exists","up"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6810","nonce":951246042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6814","nonce":951246042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6821","nonce":951246042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6818","nonce":951246042}]},"public_addr":"172.21.15.184:6810/951246042","cluster_addr":"172.21.15.184:6814/951246042","heartbeat_back_addr":"172.21.15.184:6821/951246042","heartbeat_front_addr":"172.21.15.184:6818/951246042","state":["exists","up"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6825","nonce":1941674146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6827","nonce":1941674146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6831","nonce":1941674146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6829","nonce":1941674146}]},"public_addr":"172.21.15.184:6825/1941674146","cluster_addr":"172.21.15.184:6827/1941674146","heartbeat_back_addr":"172.21.15.184:6831/1941674146","heartbeat_front_addr":"172.21.15.184:6829/1941674146","state":["exists","up"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":30,"up_from":53,"up_thru":56,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6811","nonce":48890479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6815","nonce":48890479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6823","nonce":48890479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6819","nonce":48890479}]},"public_addr":"172.21.15.184:6811/48890479","cluster_addr":"172.21.15.184:6815/48890479","heartbeat_back_addr":"172.21.15.184:6823/48890479","heartbeat_front_addr":"172.21.15.184:6819/48890479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.805494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.817444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.712083+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.739958+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.720767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.715944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.788853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.809325+0000","dead_epoch":31}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:14:30.760 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T07:14:30.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:14:31.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:31.044 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:14:30.434428+0000","last_up_change":"2024-09-13T07:13:13.343075+0000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T07:12:19.268005+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T07:12:20.513150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T07:12:28.796583+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":73,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6809","nonce":1709347595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6811","nonce":1709347595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6815","nonce":1709347595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6813","nonce":1709347595}]},"public_addr":"172.21.15.31:6809/1709347595","cluster_addr":"172.21.15.31:6811/1709347595","heartbeat_back_addr":"172.21.15.31:6815/1709347595","heartbeat_front_addr":"172.21.15.31:6813/1709347595","state":["exists","up"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6825","nonce":3300827638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6827","nonce":3300827638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6831","nonce":3300827638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6829","nonce":3300827638}]},"public_addr":"172.21.15.31:6825/3300827638","cluster_addr":"172.21.15.31:6827/3300827638","heartbeat_back_addr":"172.21.15.31:6831/3300827638","heartbeat_front_addr":"172.21.15.31:6829/3300827638","state":["exists","up"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6801","nonce":3977233096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6803","nonce":3977233096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6807","nonce":3977233096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6805","nonce":3977233096}]},"public_addr":"172.21.15.31:6801/3977233096","cluster_addr":"172.21.15.31:6803/3977233096","heartbeat_back_addr":"172.21.15.31:6807/3977233096","heartbeat_front_addr":"172.21.15.31:6805/3977233096","state":["exists","up"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6817","nonce":3290545828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6819","nonce":3290545828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6823","nonce":3290545828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6821","nonce":3290545828}]},"public_addr":"172.21.15.31:6817/3290545828","cluster_addr":"172.21.15.31:6819/3290545828","heartbeat_back_addr":"172.21.15.31:6823/3290545828","heartbeat_front_addr":"172.21.15.31:6821/3290545828","state":["exists","up"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6801","nonce":3543694628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6803","nonce":3543694628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6807","nonce":3543694628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6805","nonce":3543694628}]},"public_addr":"172.21.15.184:6801/3543694628","cluster_addr":"172.21.15.184:6803/3543694628","heartbeat_back_addr":"172.21.15.184:6807/3543694628","heartbeat_front_addr":"172.21.15.184:6805/3543694628","state":["exists","up"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6810","nonce":951246042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6814","nonce":951246042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6821","nonce":951246042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6818","nonce":951246042}]},"public_addr":"172.21.15.184:6810/951246042","cluster_addr":"172.21.15.184:6814/951246042","heartbeat_back_addr":"172.21.15.184:6821/951246042","heartbeat_front_addr":"172.21.15.184:6818/951246042","state":["exists","up"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6825","nonce":1941674146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6827","nonce":1941674146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6831","nonce":1941674146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6829","nonce":1941674146}]},"public_addr":"172.21.15.184:6825/1941674146","cluster_addr":"172.21.15.184:6827/1941674146","heartbeat_back_addr":"172.21.15.184:6831/1941674146","heartbeat_front_addr":"172.21.15.184:6829/1941674146","state":["exists","up"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":30,"up_from":53,"up_thru":56,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6811","nonce":48890479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6815","nonce":48890479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6823","nonce":48890479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6819","nonce":48890479}]},"public_addr":"172.21.15.184:6811/48890479","cluster_addr":"172.21.15.184:6815/48890479","heartbeat_back_addr":"172.21.15.184:6823/48890479","heartbeat_front_addr":"172.21.15.184:6819/48890479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.805494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.817444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.712083+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.739958+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.720767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.715944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.788853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.809325+0000","dead_epoch":31}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:14:31.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T07:14:31.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T07:14:31.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T07:14:31.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T07:14:31.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T07:14:31.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T07:14:31.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T07:14:31.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T07:14:31.306 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:31.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:14:31.308 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:31.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:14:31.310 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:31.311 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:31.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:14:31.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:14:31.314 INFO:teuthology.orchestra.run.smithi031.stdout:55834574880 2024-09-13T07:14:31.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:14:31.315 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:31.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:14:31.316 INFO:teuthology.orchestra.run.smithi031.stdout:227633266705 2024-09-13T07:14:31.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:14:31.319 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:31.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:14:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:55834574877 2024-09-13T07:14:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:55834574877 2024-09-13T07:14:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:55834574877 2024-09-13T07:14:31.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574877 for osd.2 2024-09-13T07:14:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:55834574877 2024-09-13T07:14:31.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574877 for osd.4 2024-09-13T07:14:31.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574877 for osd.0 2024-09-13T07:14:31.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574877 for osd.5 2024-09-13T07:14:31.925 INFO:teuthology.orchestra.run.smithi031.stdout:55834574877 2024-09-13T07:14:31.932 INFO:teuthology.orchestra.run.smithi031.stdout:227633266704 2024-09-13T07:14:31.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574877 for osd.1 2024-09-13T07:14:31.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 227633266705 got 227633266704 for osd.7 2024-09-13T07:14:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:55834574877 2024-09-13T07:14:31.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574877 for osd.3 2024-09-13T07:14:31.968 INFO:teuthology.orchestra.run.smithi031.stdout:55834574879 2024-09-13T07:14:31.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574879 for osd.6 2024-09-13T07:14:32.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:14:32.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:14:32.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:14:32.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:14:32.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:14:32.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:14:32.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:14:32.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:14:33.434 INFO:teuthology.orchestra.run.smithi031.stdout:55834574877 2024-09-13T07:14:33.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574877 for osd.2 2024-09-13T07:14:33.509 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:33.512 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:33.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574878 for osd.0 2024-09-13T07:14:33.526 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:33.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574878 for osd.5 2024-09-13T07:14:33.530 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:33.531 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:33.535 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:33.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574878 for osd.1 2024-09-13T07:14:33.545 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:33.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574878 for osd.4 2024-09-13T07:14:33.554 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:33.561 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:33.567 INFO:teuthology.orchestra.run.smithi031.stdout:55834574880 2024-09-13T07:14:33.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574878 for osd.3 2024-09-13T07:14:33.576 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:33.580 INFO:teuthology.orchestra.run.smithi031.stdout:227633266706 2024-09-13T07:14:33.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.6 2024-09-13T07:14:33.582 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:33.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 227633266705 got 227633266706 for osd.7 2024-09-13T07:14:33.594 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:34.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:14:34.752 INFO:teuthology.orchestra.run.smithi031.stdout:55834574878 2024-09-13T07:14:34.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574878 got 55834574878 for osd.2 2024-09-13T07:14:34.764 DEBUG:teuthology.parallel:result is None 2024-09-13T07:14:34.764 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T07:14:34.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:14:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:35.051 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:14:35.062 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-13T07:14:34.493967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16842,"ondisk_log_size":16842,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333148,"kb_used_data":11976,"kb_used_omap":20,"kb_used_meta":321003,"kb_avail":754641572,"statfs":{"total":773094113280,"available":772752969728,"internally_reserved":0,"allocated":12263424,"data_stored":6967138,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":328707328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070170"},"pg_stats":[{"pgid":"3.1f","version":"106'351","reported_seq":4067,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537193+0000","last_change":"2024-09-13T07:14:08.185500+0000","last_active":"2024-09-13T07:14:30.537193+0000","last_peered":"2024-09-13T07:14:30.537193+0000","last_clean":"2024-09-13T07:14:30.537193+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:30.537193+0000","last_undegraded":"2024-09-13T07:14:30.537193+0000","last_fullsized":"2024-09-13T07:14:30.537193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"106'377","reported_seq":4502,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506606+0000","last_change":"2024-09-13T07:14:08.147692+0000","last_active":"2024-09-13T07:14:29.506606+0000","last_peered":"2024-09-13T07:14:29.506606+0000","last_clean":"2024-09-13T07:14:29.506606+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:29.506606+0000","last_undegraded":"2024-09-13T07:14:29.506606+0000","last_fullsized":"2024-09-13T07:14:29.506606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"106'659","reported_seq":2765,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802835+0000","last_change":"2024-09-13T07:14:08.147661+0000","last_active":"2024-09-13T07:14:29.802835+0000","last_peered":"2024-09-13T07:14:29.802835+0000","last_clean":"2024-09-13T07:14:29.802835+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:29.802835+0000","last_undegraded":"2024-09-13T07:14:29.802835+0000","last_fullsized":"2024-09-13T07:14:29.802835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"106'508","reported_seq":3662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802799+0000","last_change":"2024-09-13T07:14:08.147649+0000","last_active":"2024-09-13T07:14:29.802799+0000","last_peered":"2024-09-13T07:14:29.802799+0000","last_clean":"2024-09-13T07:14:29.802799+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:29.802799+0000","last_undegraded":"2024-09-13T07:14:29.802799+0000","last_fullsized":"2024-09-13T07:14:29.802799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'1116","reported_seq":4756,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802255+0000","last_change":"2024-09-13T07:14:08.185626+0000","last_active":"2024-09-13T07:14:29.802255+0000","last_peered":"2024-09-13T07:14:29.802255+0000","last_clean":"2024-09-13T07:14:29.802255+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:29.802255+0000","last_undegraded":"2024-09-13T07:14:29.802255+0000","last_fullsized":"2024-09-13T07:14:29.802255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"101'357","reported_seq":4424,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803320+0000","last_change":"2024-09-13T07:14:08.153387+0000","last_active":"2024-09-13T07:14:29.803320+0000","last_peered":"2024-09-13T07:14:29.803320+0000","last_clean":"2024-09-13T07:14:29.803320+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:29.803320+0000","last_undegraded":"2024-09-13T07:14:29.803320+0000","last_fullsized":"2024-09-13T07:14:29.803320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"105'354","reported_seq":5316,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.431918+0000","last_change":"2024-09-13T07:14:08.255476+0000","last_active":"2024-09-13T07:14:29.431918+0000","last_peered":"2024-09-13T07:14:29.431918+0000","last_clean":"2024-09-13T07:14:29.431918+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:29.431918+0000","last_undegraded":"2024-09-13T07:14:29.431918+0000","last_fullsized":"2024-09-13T07:14:29.431918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"101'572","reported_seq":5592,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432239+0000","last_change":"2024-09-13T07:14:08.147872+0000","last_active":"2024-09-13T07:14:29.432239+0000","last_peered":"2024-09-13T07:14:29.432239+0000","last_clean":"2024-09-13T07:14:29.432239+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:29.432239+0000","last_undegraded":"2024-09-13T07:14:29.432239+0000","last_fullsized":"2024-09-13T07:14:29.432239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"106'499","reported_seq":3130,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802271+0000","last_change":"2024-09-13T07:14:08.185299+0000","last_active":"2024-09-13T07:14:29.802271+0000","last_peered":"2024-09-13T07:14:29.802271+0000","last_clean":"2024-09-13T07:14:29.802271+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:29.802271+0000","last_undegraded":"2024-09-13T07:14:29.802271+0000","last_fullsized":"2024-09-13T07:14:29.802271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"103'248","reported_seq":1373,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802691+0000","last_change":"2024-09-13T07:14:08.147385+0000","last_active":"2024-09-13T07:14:29.802691+0000","last_peered":"2024-09-13T07:14:29.802691+0000","last_clean":"2024-09-13T07:14:29.802691+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:29.802691+0000","last_undegraded":"2024-09-13T07:14:29.802691+0000","last_fullsized":"2024-09-13T07:14:29.802691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'337","reported_seq":4673,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802569+0000","last_change":"2024-09-13T07:14:08.151415+0000","last_active":"2024-09-13T07:14:29.802569+0000","last_peered":"2024-09-13T07:14:29.802569+0000","last_clean":"2024-09-13T07:14:29.802569+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:14:29.802569+0000","last_undegraded":"2024-09-13T07:14:29.802569+0000","last_fullsized":"2024-09-13T07:14:29.802569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"105'403","reported_seq":2120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803410+0000","last_change":"2024-09-13T07:14:08.153778+0000","last_active":"2024-09-13T07:14:29.803410+0000","last_peered":"2024-09-13T07:14:29.803410+0000","last_clean":"2024-09-13T07:14:29.803410+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:29.803410+0000","last_undegraded":"2024-09-13T07:14:29.803410+0000","last_fullsized":"2024-09-13T07:14:29.803410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438202+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:14:30.438202+0000","last_peered":"2024-09-13T07:14:30.438202+0000","last_clean":"2024-09-13T07:14:30.438202+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:30.438202+0000","last_undegraded":"2024-09-13T07:14:30.438202+0000","last_fullsized":"2024-09-13T07:14:30.438202+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"103'434","reported_seq":3836,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802922+0000","last_change":"2024-09-13T07:14:08.147440+0000","last_active":"2024-09-13T07:14:29.802922+0000","last_peered":"2024-09-13T07:14:29.802922+0000","last_clean":"2024-09-13T07:14:29.802922+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:29.802922+0000","last_undegraded":"2024-09-13T07:14:29.802922+0000","last_fullsized":"2024-09-13T07:14:29.802922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438237+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:14:30.438237+0000","last_peered":"2024-09-13T07:14:30.438237+0000","last_clean":"2024-09-13T07:14:30.438237+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:30.438237+0000","last_undegraded":"2024-09-13T07:14:30.438237+0000","last_fullsized":"2024-09-13T07:14:30.438237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"106'544","reported_seq":4371,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432122+0000","last_change":"2024-09-13T07:14:08.254223+0000","last_active":"2024-09-13T07:14:29.432122+0000","last_peered":"2024-09-13T07:14:29.432122+0000","last_clean":"2024-09-13T07:14:29.432122+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:29.432122+0000","last_undegraded":"2024-09-13T07:14:29.432122+0000","last_fullsized":"2024-09-13T07:14:29.432122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803432+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:14:29.803432+0000","last_peered":"2024-09-13T07:14:29.803432+0000","last_clean":"2024-09-13T07:14:29.803432+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:29.803432+0000","last_undegraded":"2024-09-13T07:14:29.803432+0000","last_fullsized":"2024-09-13T07:14:29.803432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"101'375","reported_seq":986,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432395+0000","last_change":"2024-09-13T07:14:08.148019+0000","last_active":"2024-09-13T07:14:29.432395+0000","last_peered":"2024-09-13T07:14:29.432395+0000","last_clean":"2024-09-13T07:14:29.432395+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:29.432395+0000","last_undegraded":"2024-09-13T07:14:29.432395+0000","last_fullsized":"2024-09-13T07:14:29.432395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506593+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:14:29.506593+0000","last_peered":"2024-09-13T07:14:29.506593+0000","last_clean":"2024-09-13T07:14:29.506593+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:29.506593+0000","last_undegraded":"2024-09-13T07:14:29.506593+0000","last_fullsized":"2024-09-13T07:14:29.506593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"106'386","reported_seq":7850,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802240+0000","last_change":"2024-09-13T07:14:08.185004+0000","last_active":"2024-09-13T07:14:29.802240+0000","last_peered":"2024-09-13T07:14:29.802240+0000","last_clean":"2024-09-13T07:14:29.802240+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:29.802240+0000","last_undegraded":"2024-09-13T07:14:29.802240+0000","last_fullsized":"2024-09-13T07:14:29.802240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"105'595","reported_seq":4780,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537058+0000","last_change":"2024-09-13T07:14:08.185870+0000","last_active":"2024-09-13T07:14:30.537058+0000","last_peered":"2024-09-13T07:14:30.537058+0000","last_clean":"2024-09-13T07:14:30.537058+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:30.537058+0000","last_undegraded":"2024-09-13T07:14:30.537058+0000","last_fullsized":"2024-09-13T07:14:30.537058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":243,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802501+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:14:29.802501+0000","last_peered":"2024-09-13T07:14:29.802501+0000","last_clean":"2024-09-13T07:14:29.802501+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:29.802501+0000","last_undegraded":"2024-09-13T07:14:29.802501+0000","last_fullsized":"2024-09-13T07:14:29.802501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":128,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503759+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:14:29.503759+0000","last_peered":"2024-09-13T07:14:29.503759+0000","last_clean":"2024-09-13T07:14:29.503759+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:29.503759+0000","last_undegraded":"2024-09-13T07:14:29.503759+0000","last_fullsized":"2024-09-13T07:14:29.503759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'467","reported_seq":2709,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503735+0000","last_change":"2024-09-13T07:14:08.255416+0000","last_active":"2024-09-13T07:14:29.503735+0000","last_peered":"2024-09-13T07:14:29.503735+0000","last_clean":"2024-09-13T07:14:29.503735+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:29.503735+0000","last_undegraded":"2024-09-13T07:14:29.503735+0000","last_fullsized":"2024-09-13T07:14:29.503735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802671+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:14:29.802671+0000","last_peered":"2024-09-13T07:14:29.802671+0000","last_clean":"2024-09-13T07:14:29.802671+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:14:29.802671+0000","last_undegraded":"2024-09-13T07:14:29.802671+0000","last_fullsized":"2024-09-13T07:14:29.802671+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"106'606","reported_seq":4415,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803401+0000","last_change":"2024-09-13T07:14:08.154052+0000","last_active":"2024-09-13T07:14:29.803401+0000","last_peered":"2024-09-13T07:14:29.803401+0000","last_clean":"2024-09-13T07:14:29.803401+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:29.803401+0000","last_undegraded":"2024-09-13T07:14:29.803401+0000","last_fullsized":"2024-09-13T07:14:29.803401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802669+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:14:29.802669+0000","last_peered":"2024-09-13T07:14:29.802669+0000","last_clean":"2024-09-13T07:14:29.802669+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:29.802669+0000","last_undegraded":"2024-09-13T07:14:29.802669+0000","last_fullsized":"2024-09-13T07:14:29.802669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"105'494","reported_seq":5282,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432302+0000","last_change":"2024-09-13T07:14:08.147309+0000","last_active":"2024-09-13T07:14:29.432302+0000","last_peered":"2024-09-13T07:14:29.432302+0000","last_clean":"2024-09-13T07:14:29.432302+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:29.432302+0000","last_undegraded":"2024-09-13T07:14:29.432302+0000","last_fullsized":"2024-09-13T07:14:29.432302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802464+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:14:29.802464+0000","last_peered":"2024-09-13T07:14:29.802464+0000","last_clean":"2024-09-13T07:14:29.802464+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:29.802464+0000","last_undegraded":"2024-09-13T07:14:29.802464+0000","last_fullsized":"2024-09-13T07:14:29.802464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'476","reported_seq":2374,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432192+0000","last_change":"2024-09-13T07:14:13.315962+0000","last_active":"2024-09-13T07:14:29.432192+0000","last_peered":"2024-09-13T07:14:29.432192+0000","last_clean":"2024-09-13T07:14:29.432192+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:29.432192+0000","last_undegraded":"2024-09-13T07:14:29.432192+0000","last_fullsized":"2024-09-13T07:14:29.432192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:39.120242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"104'520","reported_seq":1525,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803295+0000","last_change":"2024-09-13T07:14:08.153704+0000","last_active":"2024-09-13T07:14:29.803295+0000","last_peered":"2024-09-13T07:14:29.803295+0000","last_clean":"2024-09-13T07:14:29.803295+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:29.803295+0000","last_undegraded":"2024-09-13T07:14:29.803295+0000","last_fullsized":"2024-09-13T07:14:29.803295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"104'824","reported_seq":3683,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432245+0000","last_change":"2024-09-13T07:14:08.175514+0000","last_active":"2024-09-13T07:14:29.432245+0000","last_peered":"2024-09-13T07:14:29.432245+0000","last_clean":"2024-09-13T07:14:29.432245+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:29.432245+0000","last_undegraded":"2024-09-13T07:14:29.432245+0000","last_fullsized":"2024-09-13T07:14:29.432245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"106'861","reported_seq":3925,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432295+0000","last_change":"2024-09-13T07:14:08.147574+0000","last_active":"2024-09-13T07:14:29.432295+0000","last_peered":"2024-09-13T07:14:29.432295+0000","last_clean":"2024-09-13T07:14:29.432295+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:29.432295+0000","last_undegraded":"2024-09-13T07:14:29.432295+0000","last_fullsized":"2024-09-13T07:14:29.432295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'319","reported_seq":2404,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802698+0000","last_change":"2024-09-13T07:14:08.147077+0000","last_active":"2024-09-13T07:14:29.802698+0000","last_peered":"2024-09-13T07:14:29.802698+0000","last_clean":"2024-09-13T07:14:29.802698+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:29.802698+0000","last_undegraded":"2024-09-13T07:14:29.802698+0000","last_fullsized":"2024-09-13T07:14:29.802698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'379","reported_seq":2462,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802571+0000","last_change":"2024-09-13T07:14:08.151192+0000","last_active":"2024-09-13T07:14:29.802571+0000","last_peered":"2024-09-13T07:14:29.802571+0000","last_clean":"2024-09-13T07:14:29.802571+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:14:29.802571+0000","last_undegraded":"2024-09-13T07:14:29.802571+0000","last_fullsized":"2024-09-13T07:14:29.802571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"106'708","reported_seq":2117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802887+0000","last_change":"2024-09-13T07:14:08.151147+0000","last_active":"2024-09-13T07:14:29.802887+0000","last_peered":"2024-09-13T07:14:29.802887+0000","last_clean":"2024-09-13T07:14:29.802887+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:14:29.802887+0000","last_undegraded":"2024-09-13T07:14:29.802887+0000","last_fullsized":"2024-09-13T07:14:29.802887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"106'1061","reported_seq":7532,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802711+0000","last_change":"2024-09-13T07:14:08.151555+0000","last_active":"2024-09-13T07:14:29.802711+0000","last_peered":"2024-09-13T07:14:29.802711+0000","last_clean":"2024-09-13T07:14:29.802711+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:14:29.802711+0000","last_undegraded":"2024-09-13T07:14:29.802711+0000","last_fullsized":"2024-09-13T07:14:29.802711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"106'447","reported_seq":2522,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432275+0000","last_change":"2024-09-13T07:14:08.175489+0000","last_active":"2024-09-13T07:14:29.432275+0000","last_peered":"2024-09-13T07:14:29.432275+0000","last_clean":"2024-09-13T07:14:29.432275+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:29.432275+0000","last_undegraded":"2024-09-13T07:14:29.432275+0000","last_fullsized":"2024-09-13T07:14:29.432275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"104'464","reported_seq":3904,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802525+0000","last_change":"2024-09-13T07:14:08.150089+0000","last_active":"2024-09-13T07:14:29.802525+0000","last_peered":"2024-09-13T07:14:29.802525+0000","last_clean":"2024-09-13T07:14:29.802525+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:14:29.802525+0000","last_undegraded":"2024-09-13T07:14:29.802525+0000","last_fullsized":"2024-09-13T07:14:29.802525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"105'291","reported_seq":1892,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802205+0000","last_change":"2024-09-13T07:14:08.185172+0000","last_active":"2024-09-13T07:14:29.802205+0000","last_peered":"2024-09-13T07:14:29.802205+0000","last_clean":"2024-09-13T07:14:29.802205+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:29.802205+0000","last_undegraded":"2024-09-13T07:14:29.802205+0000","last_fullsized":"2024-09-13T07:14:29.802205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"103'690","reported_seq":4171,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802547+0000","last_change":"2024-09-13T07:14:08.150123+0000","last_active":"2024-09-13T07:14:29.802547+0000","last_peered":"2024-09-13T07:14:29.802547+0000","last_clean":"2024-09-13T07:14:29.802547+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:14:29.802547+0000","last_undegraded":"2024-09-13T07:14:29.802547+0000","last_fullsized":"2024-09-13T07:14:29.802547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":16722,"ondisk_log_size":16722,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266706,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38428,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94333412,"statfs":{"total":96636764160,"available":96597413888,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.63},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.5},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.436}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.80500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.751},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28512,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343328,"statfs":{"total":96636764160,"available":96607567872,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.433},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.34599999999999997}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.37}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64596,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307244,"statfs":{"total":96636764160,"available":96570617856,"internally_reserved":0,"allocated":1511424,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.36799999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1507328,"data_stored":718931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.308}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.376},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.441},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.504}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.442},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.745},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.70999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.377}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.625},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:14:35.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:14:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:35.351 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:14:35.361 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-13T07:14:34.493967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16842,"ondisk_log_size":16842,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333148,"kb_used_data":11976,"kb_used_omap":20,"kb_used_meta":321003,"kb_avail":754641572,"statfs":{"total":773094113280,"available":772752969728,"internally_reserved":0,"allocated":12263424,"data_stored":6967138,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":328707328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070170"},"pg_stats":[{"pgid":"3.1f","version":"106'351","reported_seq":4067,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537193+0000","last_change":"2024-09-13T07:14:08.185500+0000","last_active":"2024-09-13T07:14:30.537193+0000","last_peered":"2024-09-13T07:14:30.537193+0000","last_clean":"2024-09-13T07:14:30.537193+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:30.537193+0000","last_undegraded":"2024-09-13T07:14:30.537193+0000","last_fullsized":"2024-09-13T07:14:30.537193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"106'377","reported_seq":4502,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506606+0000","last_change":"2024-09-13T07:14:08.147692+0000","last_active":"2024-09-13T07:14:29.506606+0000","last_peered":"2024-09-13T07:14:29.506606+0000","last_clean":"2024-09-13T07:14:29.506606+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:29.506606+0000","last_undegraded":"2024-09-13T07:14:29.506606+0000","last_fullsized":"2024-09-13T07:14:29.506606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"106'659","reported_seq":2765,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802835+0000","last_change":"2024-09-13T07:14:08.147661+0000","last_active":"2024-09-13T07:14:29.802835+0000","last_peered":"2024-09-13T07:14:29.802835+0000","last_clean":"2024-09-13T07:14:29.802835+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:29.802835+0000","last_undegraded":"2024-09-13T07:14:29.802835+0000","last_fullsized":"2024-09-13T07:14:29.802835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"106'508","reported_seq":3662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802799+0000","last_change":"2024-09-13T07:14:08.147649+0000","last_active":"2024-09-13T07:14:29.802799+0000","last_peered":"2024-09-13T07:14:29.802799+0000","last_clean":"2024-09-13T07:14:29.802799+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:29.802799+0000","last_undegraded":"2024-09-13T07:14:29.802799+0000","last_fullsized":"2024-09-13T07:14:29.802799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'1116","reported_seq":4756,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802255+0000","last_change":"2024-09-13T07:14:08.185626+0000","last_active":"2024-09-13T07:14:29.802255+0000","last_peered":"2024-09-13T07:14:29.802255+0000","last_clean":"2024-09-13T07:14:29.802255+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:29.802255+0000","last_undegraded":"2024-09-13T07:14:29.802255+0000","last_fullsized":"2024-09-13T07:14:29.802255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"101'357","reported_seq":4424,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803320+0000","last_change":"2024-09-13T07:14:08.153387+0000","last_active":"2024-09-13T07:14:29.803320+0000","last_peered":"2024-09-13T07:14:29.803320+0000","last_clean":"2024-09-13T07:14:29.803320+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:29.803320+0000","last_undegraded":"2024-09-13T07:14:29.803320+0000","last_fullsized":"2024-09-13T07:14:29.803320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"105'354","reported_seq":5316,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.431918+0000","last_change":"2024-09-13T07:14:08.255476+0000","last_active":"2024-09-13T07:14:29.431918+0000","last_peered":"2024-09-13T07:14:29.431918+0000","last_clean":"2024-09-13T07:14:29.431918+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:29.431918+0000","last_undegraded":"2024-09-13T07:14:29.431918+0000","last_fullsized":"2024-09-13T07:14:29.431918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"101'572","reported_seq":5592,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432239+0000","last_change":"2024-09-13T07:14:08.147872+0000","last_active":"2024-09-13T07:14:29.432239+0000","last_peered":"2024-09-13T07:14:29.432239+0000","last_clean":"2024-09-13T07:14:29.432239+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:29.432239+0000","last_undegraded":"2024-09-13T07:14:29.432239+0000","last_fullsized":"2024-09-13T07:14:29.432239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"106'499","reported_seq":3130,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802271+0000","last_change":"2024-09-13T07:14:08.185299+0000","last_active":"2024-09-13T07:14:29.802271+0000","last_peered":"2024-09-13T07:14:29.802271+0000","last_clean":"2024-09-13T07:14:29.802271+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:29.802271+0000","last_undegraded":"2024-09-13T07:14:29.802271+0000","last_fullsized":"2024-09-13T07:14:29.802271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"103'248","reported_seq":1373,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802691+0000","last_change":"2024-09-13T07:14:08.147385+0000","last_active":"2024-09-13T07:14:29.802691+0000","last_peered":"2024-09-13T07:14:29.802691+0000","last_clean":"2024-09-13T07:14:29.802691+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:29.802691+0000","last_undegraded":"2024-09-13T07:14:29.802691+0000","last_fullsized":"2024-09-13T07:14:29.802691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'337","reported_seq":4673,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802569+0000","last_change":"2024-09-13T07:14:08.151415+0000","last_active":"2024-09-13T07:14:29.802569+0000","last_peered":"2024-09-13T07:14:29.802569+0000","last_clean":"2024-09-13T07:14:29.802569+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:14:29.802569+0000","last_undegraded":"2024-09-13T07:14:29.802569+0000","last_fullsized":"2024-09-13T07:14:29.802569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"105'403","reported_seq":2120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803410+0000","last_change":"2024-09-13T07:14:08.153778+0000","last_active":"2024-09-13T07:14:29.803410+0000","last_peered":"2024-09-13T07:14:29.803410+0000","last_clean":"2024-09-13T07:14:29.803410+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:29.803410+0000","last_undegraded":"2024-09-13T07:14:29.803410+0000","last_fullsized":"2024-09-13T07:14:29.803410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438202+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:14:30.438202+0000","last_peered":"2024-09-13T07:14:30.438202+0000","last_clean":"2024-09-13T07:14:30.438202+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:30.438202+0000","last_undegraded":"2024-09-13T07:14:30.438202+0000","last_fullsized":"2024-09-13T07:14:30.438202+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"103'434","reported_seq":3836,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802922+0000","last_change":"2024-09-13T07:14:08.147440+0000","last_active":"2024-09-13T07:14:29.802922+0000","last_peered":"2024-09-13T07:14:29.802922+0000","last_clean":"2024-09-13T07:14:29.802922+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:29.802922+0000","last_undegraded":"2024-09-13T07:14:29.802922+0000","last_fullsized":"2024-09-13T07:14:29.802922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438237+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:14:30.438237+0000","last_peered":"2024-09-13T07:14:30.438237+0000","last_clean":"2024-09-13T07:14:30.438237+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:30.438237+0000","last_undegraded":"2024-09-13T07:14:30.438237+0000","last_fullsized":"2024-09-13T07:14:30.438237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"106'544","reported_seq":4371,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432122+0000","last_change":"2024-09-13T07:14:08.254223+0000","last_active":"2024-09-13T07:14:29.432122+0000","last_peered":"2024-09-13T07:14:29.432122+0000","last_clean":"2024-09-13T07:14:29.432122+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:29.432122+0000","last_undegraded":"2024-09-13T07:14:29.432122+0000","last_fullsized":"2024-09-13T07:14:29.432122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803432+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:14:29.803432+0000","last_peered":"2024-09-13T07:14:29.803432+0000","last_clean":"2024-09-13T07:14:29.803432+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:29.803432+0000","last_undegraded":"2024-09-13T07:14:29.803432+0000","last_fullsized":"2024-09-13T07:14:29.803432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"101'375","reported_seq":986,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432395+0000","last_change":"2024-09-13T07:14:08.148019+0000","last_active":"2024-09-13T07:14:29.432395+0000","last_peered":"2024-09-13T07:14:29.432395+0000","last_clean":"2024-09-13T07:14:29.432395+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:29.432395+0000","last_undegraded":"2024-09-13T07:14:29.432395+0000","last_fullsized":"2024-09-13T07:14:29.432395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506593+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:14:29.506593+0000","last_peered":"2024-09-13T07:14:29.506593+0000","last_clean":"2024-09-13T07:14:29.506593+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:29.506593+0000","last_undegraded":"2024-09-13T07:14:29.506593+0000","last_fullsized":"2024-09-13T07:14:29.506593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"106'386","reported_seq":7850,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802240+0000","last_change":"2024-09-13T07:14:08.185004+0000","last_active":"2024-09-13T07:14:29.802240+0000","last_peered":"2024-09-13T07:14:29.802240+0000","last_clean":"2024-09-13T07:14:29.802240+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:29.802240+0000","last_undegraded":"2024-09-13T07:14:29.802240+0000","last_fullsized":"2024-09-13T07:14:29.802240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"105'595","reported_seq":4780,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537058+0000","last_change":"2024-09-13T07:14:08.185870+0000","last_active":"2024-09-13T07:14:30.537058+0000","last_peered":"2024-09-13T07:14:30.537058+0000","last_clean":"2024-09-13T07:14:30.537058+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:30.537058+0000","last_undegraded":"2024-09-13T07:14:30.537058+0000","last_fullsized":"2024-09-13T07:14:30.537058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":243,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802501+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:14:29.802501+0000","last_peered":"2024-09-13T07:14:29.802501+0000","last_clean":"2024-09-13T07:14:29.802501+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:29.802501+0000","last_undegraded":"2024-09-13T07:14:29.802501+0000","last_fullsized":"2024-09-13T07:14:29.802501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":128,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503759+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:14:29.503759+0000","last_peered":"2024-09-13T07:14:29.503759+0000","last_clean":"2024-09-13T07:14:29.503759+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:29.503759+0000","last_undegraded":"2024-09-13T07:14:29.503759+0000","last_fullsized":"2024-09-13T07:14:29.503759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'467","reported_seq":2709,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503735+0000","last_change":"2024-09-13T07:14:08.255416+0000","last_active":"2024-09-13T07:14:29.503735+0000","last_peered":"2024-09-13T07:14:29.503735+0000","last_clean":"2024-09-13T07:14:29.503735+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:29.503735+0000","last_undegraded":"2024-09-13T07:14:29.503735+0000","last_fullsized":"2024-09-13T07:14:29.503735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802671+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:14:29.802671+0000","last_peered":"2024-09-13T07:14:29.802671+0000","last_clean":"2024-09-13T07:14:29.802671+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:14:29.802671+0000","last_undegraded":"2024-09-13T07:14:29.802671+0000","last_fullsized":"2024-09-13T07:14:29.802671+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"106'606","reported_seq":4415,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803401+0000","last_change":"2024-09-13T07:14:08.154052+0000","last_active":"2024-09-13T07:14:29.803401+0000","last_peered":"2024-09-13T07:14:29.803401+0000","last_clean":"2024-09-13T07:14:29.803401+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:29.803401+0000","last_undegraded":"2024-09-13T07:14:29.803401+0000","last_fullsized":"2024-09-13T07:14:29.803401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802669+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:14:29.802669+0000","last_peered":"2024-09-13T07:14:29.802669+0000","last_clean":"2024-09-13T07:14:29.802669+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:29.802669+0000","last_undegraded":"2024-09-13T07:14:29.802669+0000","last_fullsized":"2024-09-13T07:14:29.802669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"105'494","reported_seq":5282,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432302+0000","last_change":"2024-09-13T07:14:08.147309+0000","last_active":"2024-09-13T07:14:29.432302+0000","last_peered":"2024-09-13T07:14:29.432302+0000","last_clean":"2024-09-13T07:14:29.432302+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:29.432302+0000","last_undegraded":"2024-09-13T07:14:29.432302+0000","last_fullsized":"2024-09-13T07:14:29.432302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802464+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:14:29.802464+0000","last_peered":"2024-09-13T07:14:29.802464+0000","last_clean":"2024-09-13T07:14:29.802464+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:29.802464+0000","last_undegraded":"2024-09-13T07:14:29.802464+0000","last_fullsized":"2024-09-13T07:14:29.802464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'476","reported_seq":2374,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432192+0000","last_change":"2024-09-13T07:14:13.315962+0000","last_active":"2024-09-13T07:14:29.432192+0000","last_peered":"2024-09-13T07:14:29.432192+0000","last_clean":"2024-09-13T07:14:29.432192+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:29.432192+0000","last_undegraded":"2024-09-13T07:14:29.432192+0000","last_fullsized":"2024-09-13T07:14:29.432192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:39.120242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"104'520","reported_seq":1525,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803295+0000","last_change":"2024-09-13T07:14:08.153704+0000","last_active":"2024-09-13T07:14:29.803295+0000","last_peered":"2024-09-13T07:14:29.803295+0000","last_clean":"2024-09-13T07:14:29.803295+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:29.803295+0000","last_undegraded":"2024-09-13T07:14:29.803295+0000","last_fullsized":"2024-09-13T07:14:29.803295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"104'824","reported_seq":3683,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432245+0000","last_change":"2024-09-13T07:14:08.175514+0000","last_active":"2024-09-13T07:14:29.432245+0000","last_peered":"2024-09-13T07:14:29.432245+0000","last_clean":"2024-09-13T07:14:29.432245+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:29.432245+0000","last_undegraded":"2024-09-13T07:14:29.432245+0000","last_fullsized":"2024-09-13T07:14:29.432245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"106'861","reported_seq":3925,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432295+0000","last_change":"2024-09-13T07:14:08.147574+0000","last_active":"2024-09-13T07:14:29.432295+0000","last_peered":"2024-09-13T07:14:29.432295+0000","last_clean":"2024-09-13T07:14:29.432295+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:29.432295+0000","last_undegraded":"2024-09-13T07:14:29.432295+0000","last_fullsized":"2024-09-13T07:14:29.432295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'319","reported_seq":2404,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802698+0000","last_change":"2024-09-13T07:14:08.147077+0000","last_active":"2024-09-13T07:14:29.802698+0000","last_peered":"2024-09-13T07:14:29.802698+0000","last_clean":"2024-09-13T07:14:29.802698+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:29.802698+0000","last_undegraded":"2024-09-13T07:14:29.802698+0000","last_fullsized":"2024-09-13T07:14:29.802698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'379","reported_seq":2462,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802571+0000","last_change":"2024-09-13T07:14:08.151192+0000","last_active":"2024-09-13T07:14:29.802571+0000","last_peered":"2024-09-13T07:14:29.802571+0000","last_clean":"2024-09-13T07:14:29.802571+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:14:29.802571+0000","last_undegraded":"2024-09-13T07:14:29.802571+0000","last_fullsized":"2024-09-13T07:14:29.802571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"106'708","reported_seq":2117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802887+0000","last_change":"2024-09-13T07:14:08.151147+0000","last_active":"2024-09-13T07:14:29.802887+0000","last_peered":"2024-09-13T07:14:29.802887+0000","last_clean":"2024-09-13T07:14:29.802887+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:14:29.802887+0000","last_undegraded":"2024-09-13T07:14:29.802887+0000","last_fullsized":"2024-09-13T07:14:29.802887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"106'1061","reported_seq":7532,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802711+0000","last_change":"2024-09-13T07:14:08.151555+0000","last_active":"2024-09-13T07:14:29.802711+0000","last_peered":"2024-09-13T07:14:29.802711+0000","last_clean":"2024-09-13T07:14:29.802711+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:14:29.802711+0000","last_undegraded":"2024-09-13T07:14:29.802711+0000","last_fullsized":"2024-09-13T07:14:29.802711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"106'447","reported_seq":2522,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432275+0000","last_change":"2024-09-13T07:14:08.175489+0000","last_active":"2024-09-13T07:14:29.432275+0000","last_peered":"2024-09-13T07:14:29.432275+0000","last_clean":"2024-09-13T07:14:29.432275+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:29.432275+0000","last_undegraded":"2024-09-13T07:14:29.432275+0000","last_fullsized":"2024-09-13T07:14:29.432275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"104'464","reported_seq":3904,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802525+0000","last_change":"2024-09-13T07:14:08.150089+0000","last_active":"2024-09-13T07:14:29.802525+0000","last_peered":"2024-09-13T07:14:29.802525+0000","last_clean":"2024-09-13T07:14:29.802525+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:14:29.802525+0000","last_undegraded":"2024-09-13T07:14:29.802525+0000","last_fullsized":"2024-09-13T07:14:29.802525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"105'291","reported_seq":1892,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802205+0000","last_change":"2024-09-13T07:14:08.185172+0000","last_active":"2024-09-13T07:14:29.802205+0000","last_peered":"2024-09-13T07:14:29.802205+0000","last_clean":"2024-09-13T07:14:29.802205+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:29.802205+0000","last_undegraded":"2024-09-13T07:14:29.802205+0000","last_fullsized":"2024-09-13T07:14:29.802205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"103'690","reported_seq":4171,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802547+0000","last_change":"2024-09-13T07:14:08.150123+0000","last_active":"2024-09-13T07:14:29.802547+0000","last_peered":"2024-09-13T07:14:29.802547+0000","last_clean":"2024-09-13T07:14:29.802547+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:14:29.802547+0000","last_undegraded":"2024-09-13T07:14:29.802547+0000","last_fullsized":"2024-09-13T07:14:29.802547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":16722,"ondisk_log_size":16722,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266706,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38428,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94333412,"statfs":{"total":96636764160,"available":96597413888,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.63},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.5},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.436}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.80500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.751},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28512,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343328,"statfs":{"total":96636764160,"available":96607567872,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.433},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.34599999999999997}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.37}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64596,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307244,"statfs":{"total":96636764160,"available":96570617856,"internally_reserved":0,"allocated":1511424,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.36799999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1507328,"data_stored":718931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.308}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.376},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.441},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.504}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.442},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.745},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.70999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.377}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.625},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:14:35.365 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T07:14:35.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T07:14:35.670 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T07:14:35.681 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T07:14:35.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:14:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:35.968 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:14:35.979 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-13T07:14:34.493967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16842,"ondisk_log_size":16842,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333148,"kb_used_data":11976,"kb_used_omap":20,"kb_used_meta":321003,"kb_avail":754641572,"statfs":{"total":773094113280,"available":772752969728,"internally_reserved":0,"allocated":12263424,"data_stored":6967138,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":328707328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070170"},"pg_stats":[{"pgid":"3.1f","version":"106'351","reported_seq":4067,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537193+0000","last_change":"2024-09-13T07:14:08.185500+0000","last_active":"2024-09-13T07:14:30.537193+0000","last_peered":"2024-09-13T07:14:30.537193+0000","last_clean":"2024-09-13T07:14:30.537193+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:30.537193+0000","last_undegraded":"2024-09-13T07:14:30.537193+0000","last_fullsized":"2024-09-13T07:14:30.537193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"106'377","reported_seq":4502,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506606+0000","last_change":"2024-09-13T07:14:08.147692+0000","last_active":"2024-09-13T07:14:29.506606+0000","last_peered":"2024-09-13T07:14:29.506606+0000","last_clean":"2024-09-13T07:14:29.506606+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:29.506606+0000","last_undegraded":"2024-09-13T07:14:29.506606+0000","last_fullsized":"2024-09-13T07:14:29.506606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"106'659","reported_seq":2765,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802835+0000","last_change":"2024-09-13T07:14:08.147661+0000","last_active":"2024-09-13T07:14:29.802835+0000","last_peered":"2024-09-13T07:14:29.802835+0000","last_clean":"2024-09-13T07:14:29.802835+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:29.802835+0000","last_undegraded":"2024-09-13T07:14:29.802835+0000","last_fullsized":"2024-09-13T07:14:29.802835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"106'508","reported_seq":3662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802799+0000","last_change":"2024-09-13T07:14:08.147649+0000","last_active":"2024-09-13T07:14:29.802799+0000","last_peered":"2024-09-13T07:14:29.802799+0000","last_clean":"2024-09-13T07:14:29.802799+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:29.802799+0000","last_undegraded":"2024-09-13T07:14:29.802799+0000","last_fullsized":"2024-09-13T07:14:29.802799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'1116","reported_seq":4756,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802255+0000","last_change":"2024-09-13T07:14:08.185626+0000","last_active":"2024-09-13T07:14:29.802255+0000","last_peered":"2024-09-13T07:14:29.802255+0000","last_clean":"2024-09-13T07:14:29.802255+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:29.802255+0000","last_undegraded":"2024-09-13T07:14:29.802255+0000","last_fullsized":"2024-09-13T07:14:29.802255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"101'357","reported_seq":4424,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803320+0000","last_change":"2024-09-13T07:14:08.153387+0000","last_active":"2024-09-13T07:14:29.803320+0000","last_peered":"2024-09-13T07:14:29.803320+0000","last_clean":"2024-09-13T07:14:29.803320+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:29.803320+0000","last_undegraded":"2024-09-13T07:14:29.803320+0000","last_fullsized":"2024-09-13T07:14:29.803320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"105'354","reported_seq":5316,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.431918+0000","last_change":"2024-09-13T07:14:08.255476+0000","last_active":"2024-09-13T07:14:29.431918+0000","last_peered":"2024-09-13T07:14:29.431918+0000","last_clean":"2024-09-13T07:14:29.431918+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:29.431918+0000","last_undegraded":"2024-09-13T07:14:29.431918+0000","last_fullsized":"2024-09-13T07:14:29.431918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"101'572","reported_seq":5592,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432239+0000","last_change":"2024-09-13T07:14:08.147872+0000","last_active":"2024-09-13T07:14:29.432239+0000","last_peered":"2024-09-13T07:14:29.432239+0000","last_clean":"2024-09-13T07:14:29.432239+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:29.432239+0000","last_undegraded":"2024-09-13T07:14:29.432239+0000","last_fullsized":"2024-09-13T07:14:29.432239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"106'499","reported_seq":3130,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802271+0000","last_change":"2024-09-13T07:14:08.185299+0000","last_active":"2024-09-13T07:14:29.802271+0000","last_peered":"2024-09-13T07:14:29.802271+0000","last_clean":"2024-09-13T07:14:29.802271+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:29.802271+0000","last_undegraded":"2024-09-13T07:14:29.802271+0000","last_fullsized":"2024-09-13T07:14:29.802271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"103'248","reported_seq":1373,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802691+0000","last_change":"2024-09-13T07:14:08.147385+0000","last_active":"2024-09-13T07:14:29.802691+0000","last_peered":"2024-09-13T07:14:29.802691+0000","last_clean":"2024-09-13T07:14:29.802691+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:29.802691+0000","last_undegraded":"2024-09-13T07:14:29.802691+0000","last_fullsized":"2024-09-13T07:14:29.802691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'337","reported_seq":4673,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802569+0000","last_change":"2024-09-13T07:14:08.151415+0000","last_active":"2024-09-13T07:14:29.802569+0000","last_peered":"2024-09-13T07:14:29.802569+0000","last_clean":"2024-09-13T07:14:29.802569+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:14:29.802569+0000","last_undegraded":"2024-09-13T07:14:29.802569+0000","last_fullsized":"2024-09-13T07:14:29.802569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"105'403","reported_seq":2120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803410+0000","last_change":"2024-09-13T07:14:08.153778+0000","last_active":"2024-09-13T07:14:29.803410+0000","last_peered":"2024-09-13T07:14:29.803410+0000","last_clean":"2024-09-13T07:14:29.803410+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:29.803410+0000","last_undegraded":"2024-09-13T07:14:29.803410+0000","last_fullsized":"2024-09-13T07:14:29.803410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438202+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:14:30.438202+0000","last_peered":"2024-09-13T07:14:30.438202+0000","last_clean":"2024-09-13T07:14:30.438202+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:30.438202+0000","last_undegraded":"2024-09-13T07:14:30.438202+0000","last_fullsized":"2024-09-13T07:14:30.438202+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"103'434","reported_seq":3836,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802922+0000","last_change":"2024-09-13T07:14:08.147440+0000","last_active":"2024-09-13T07:14:29.802922+0000","last_peered":"2024-09-13T07:14:29.802922+0000","last_clean":"2024-09-13T07:14:29.802922+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:29.802922+0000","last_undegraded":"2024-09-13T07:14:29.802922+0000","last_fullsized":"2024-09-13T07:14:29.802922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438237+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:14:30.438237+0000","last_peered":"2024-09-13T07:14:30.438237+0000","last_clean":"2024-09-13T07:14:30.438237+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:30.438237+0000","last_undegraded":"2024-09-13T07:14:30.438237+0000","last_fullsized":"2024-09-13T07:14:30.438237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"106'544","reported_seq":4371,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432122+0000","last_change":"2024-09-13T07:14:08.254223+0000","last_active":"2024-09-13T07:14:29.432122+0000","last_peered":"2024-09-13T07:14:29.432122+0000","last_clean":"2024-09-13T07:14:29.432122+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:29.432122+0000","last_undegraded":"2024-09-13T07:14:29.432122+0000","last_fullsized":"2024-09-13T07:14:29.432122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803432+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:14:29.803432+0000","last_peered":"2024-09-13T07:14:29.803432+0000","last_clean":"2024-09-13T07:14:29.803432+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:29.803432+0000","last_undegraded":"2024-09-13T07:14:29.803432+0000","last_fullsized":"2024-09-13T07:14:29.803432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"101'375","reported_seq":986,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432395+0000","last_change":"2024-09-13T07:14:08.148019+0000","last_active":"2024-09-13T07:14:29.432395+0000","last_peered":"2024-09-13T07:14:29.432395+0000","last_clean":"2024-09-13T07:14:29.432395+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:29.432395+0000","last_undegraded":"2024-09-13T07:14:29.432395+0000","last_fullsized":"2024-09-13T07:14:29.432395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506593+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:14:29.506593+0000","last_peered":"2024-09-13T07:14:29.506593+0000","last_clean":"2024-09-13T07:14:29.506593+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:29.506593+0000","last_undegraded":"2024-09-13T07:14:29.506593+0000","last_fullsized":"2024-09-13T07:14:29.506593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"106'386","reported_seq":7850,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802240+0000","last_change":"2024-09-13T07:14:08.185004+0000","last_active":"2024-09-13T07:14:29.802240+0000","last_peered":"2024-09-13T07:14:29.802240+0000","last_clean":"2024-09-13T07:14:29.802240+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:29.802240+0000","last_undegraded":"2024-09-13T07:14:29.802240+0000","last_fullsized":"2024-09-13T07:14:29.802240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"105'595","reported_seq":4780,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537058+0000","last_change":"2024-09-13T07:14:08.185870+0000","last_active":"2024-09-13T07:14:30.537058+0000","last_peered":"2024-09-13T07:14:30.537058+0000","last_clean":"2024-09-13T07:14:30.537058+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:30.537058+0000","last_undegraded":"2024-09-13T07:14:30.537058+0000","last_fullsized":"2024-09-13T07:14:30.537058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":243,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802501+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:14:29.802501+0000","last_peered":"2024-09-13T07:14:29.802501+0000","last_clean":"2024-09-13T07:14:29.802501+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:29.802501+0000","last_undegraded":"2024-09-13T07:14:29.802501+0000","last_fullsized":"2024-09-13T07:14:29.802501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":128,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503759+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:14:29.503759+0000","last_peered":"2024-09-13T07:14:29.503759+0000","last_clean":"2024-09-13T07:14:29.503759+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:29.503759+0000","last_undegraded":"2024-09-13T07:14:29.503759+0000","last_fullsized":"2024-09-13T07:14:29.503759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'467","reported_seq":2709,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503735+0000","last_change":"2024-09-13T07:14:08.255416+0000","last_active":"2024-09-13T07:14:29.503735+0000","last_peered":"2024-09-13T07:14:29.503735+0000","last_clean":"2024-09-13T07:14:29.503735+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:29.503735+0000","last_undegraded":"2024-09-13T07:14:29.503735+0000","last_fullsized":"2024-09-13T07:14:29.503735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802671+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:14:29.802671+0000","last_peered":"2024-09-13T07:14:29.802671+0000","last_clean":"2024-09-13T07:14:29.802671+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:14:29.802671+0000","last_undegraded":"2024-09-13T07:14:29.802671+0000","last_fullsized":"2024-09-13T07:14:29.802671+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"106'606","reported_seq":4415,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803401+0000","last_change":"2024-09-13T07:14:08.154052+0000","last_active":"2024-09-13T07:14:29.803401+0000","last_peered":"2024-09-13T07:14:29.803401+0000","last_clean":"2024-09-13T07:14:29.803401+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:29.803401+0000","last_undegraded":"2024-09-13T07:14:29.803401+0000","last_fullsized":"2024-09-13T07:14:29.803401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802669+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:14:29.802669+0000","last_peered":"2024-09-13T07:14:29.802669+0000","last_clean":"2024-09-13T07:14:29.802669+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:29.802669+0000","last_undegraded":"2024-09-13T07:14:29.802669+0000","last_fullsized":"2024-09-13T07:14:29.802669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"105'494","reported_seq":5282,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432302+0000","last_change":"2024-09-13T07:14:08.147309+0000","last_active":"2024-09-13T07:14:29.432302+0000","last_peered":"2024-09-13T07:14:29.432302+0000","last_clean":"2024-09-13T07:14:29.432302+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:29.432302+0000","last_undegraded":"2024-09-13T07:14:29.432302+0000","last_fullsized":"2024-09-13T07:14:29.432302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802464+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:14:29.802464+0000","last_peered":"2024-09-13T07:14:29.802464+0000","last_clean":"2024-09-13T07:14:29.802464+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:29.802464+0000","last_undegraded":"2024-09-13T07:14:29.802464+0000","last_fullsized":"2024-09-13T07:14:29.802464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'476","reported_seq":2374,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432192+0000","last_change":"2024-09-13T07:14:13.315962+0000","last_active":"2024-09-13T07:14:29.432192+0000","last_peered":"2024-09-13T07:14:29.432192+0000","last_clean":"2024-09-13T07:14:29.432192+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:29.432192+0000","last_undegraded":"2024-09-13T07:14:29.432192+0000","last_fullsized":"2024-09-13T07:14:29.432192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:39.120242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"104'520","reported_seq":1525,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803295+0000","last_change":"2024-09-13T07:14:08.153704+0000","last_active":"2024-09-13T07:14:29.803295+0000","last_peered":"2024-09-13T07:14:29.803295+0000","last_clean":"2024-09-13T07:14:29.803295+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:29.803295+0000","last_undegraded":"2024-09-13T07:14:29.803295+0000","last_fullsized":"2024-09-13T07:14:29.803295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"104'824","reported_seq":3683,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432245+0000","last_change":"2024-09-13T07:14:08.175514+0000","last_active":"2024-09-13T07:14:29.432245+0000","last_peered":"2024-09-13T07:14:29.432245+0000","last_clean":"2024-09-13T07:14:29.432245+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:29.432245+0000","last_undegraded":"2024-09-13T07:14:29.432245+0000","last_fullsized":"2024-09-13T07:14:29.432245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"106'861","reported_seq":3925,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432295+0000","last_change":"2024-09-13T07:14:08.147574+0000","last_active":"2024-09-13T07:14:29.432295+0000","last_peered":"2024-09-13T07:14:29.432295+0000","last_clean":"2024-09-13T07:14:29.432295+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:29.432295+0000","last_undegraded":"2024-09-13T07:14:29.432295+0000","last_fullsized":"2024-09-13T07:14:29.432295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'319","reported_seq":2404,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802698+0000","last_change":"2024-09-13T07:14:08.147077+0000","last_active":"2024-09-13T07:14:29.802698+0000","last_peered":"2024-09-13T07:14:29.802698+0000","last_clean":"2024-09-13T07:14:29.802698+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:29.802698+0000","last_undegraded":"2024-09-13T07:14:29.802698+0000","last_fullsized":"2024-09-13T07:14:29.802698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'379","reported_seq":2462,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802571+0000","last_change":"2024-09-13T07:14:08.151192+0000","last_active":"2024-09-13T07:14:29.802571+0000","last_peered":"2024-09-13T07:14:29.802571+0000","last_clean":"2024-09-13T07:14:29.802571+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:14:29.802571+0000","last_undegraded":"2024-09-13T07:14:29.802571+0000","last_fullsized":"2024-09-13T07:14:29.802571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"106'708","reported_seq":2117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802887+0000","last_change":"2024-09-13T07:14:08.151147+0000","last_active":"2024-09-13T07:14:29.802887+0000","last_peered":"2024-09-13T07:14:29.802887+0000","last_clean":"2024-09-13T07:14:29.802887+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:14:29.802887+0000","last_undegraded":"2024-09-13T07:14:29.802887+0000","last_fullsized":"2024-09-13T07:14:29.802887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"106'1061","reported_seq":7532,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802711+0000","last_change":"2024-09-13T07:14:08.151555+0000","last_active":"2024-09-13T07:14:29.802711+0000","last_peered":"2024-09-13T07:14:29.802711+0000","last_clean":"2024-09-13T07:14:29.802711+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:14:29.802711+0000","last_undegraded":"2024-09-13T07:14:29.802711+0000","last_fullsized":"2024-09-13T07:14:29.802711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"106'447","reported_seq":2522,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432275+0000","last_change":"2024-09-13T07:14:08.175489+0000","last_active":"2024-09-13T07:14:29.432275+0000","last_peered":"2024-09-13T07:14:29.432275+0000","last_clean":"2024-09-13T07:14:29.432275+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:29.432275+0000","last_undegraded":"2024-09-13T07:14:29.432275+0000","last_fullsized":"2024-09-13T07:14:29.432275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"104'464","reported_seq":3904,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802525+0000","last_change":"2024-09-13T07:14:08.150089+0000","last_active":"2024-09-13T07:14:29.802525+0000","last_peered":"2024-09-13T07:14:29.802525+0000","last_clean":"2024-09-13T07:14:29.802525+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:14:29.802525+0000","last_undegraded":"2024-09-13T07:14:29.802525+0000","last_fullsized":"2024-09-13T07:14:29.802525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"105'291","reported_seq":1892,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802205+0000","last_change":"2024-09-13T07:14:08.185172+0000","last_active":"2024-09-13T07:14:29.802205+0000","last_peered":"2024-09-13T07:14:29.802205+0000","last_clean":"2024-09-13T07:14:29.802205+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:29.802205+0000","last_undegraded":"2024-09-13T07:14:29.802205+0000","last_fullsized":"2024-09-13T07:14:29.802205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"103'690","reported_seq":4171,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802547+0000","last_change":"2024-09-13T07:14:08.150123+0000","last_active":"2024-09-13T07:14:29.802547+0000","last_peered":"2024-09-13T07:14:29.802547+0000","last_clean":"2024-09-13T07:14:29.802547+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:14:29.802547+0000","last_undegraded":"2024-09-13T07:14:29.802547+0000","last_fullsized":"2024-09-13T07:14:29.802547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":16722,"ondisk_log_size":16722,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266706,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38428,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94333412,"statfs":{"total":96636764160,"available":96597413888,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.63},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.5},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.436}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.80500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.751},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28512,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343328,"statfs":{"total":96636764160,"available":96607567872,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.433},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.34599999999999997}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.37}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64596,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307244,"statfs":{"total":96636764160,"available":96570617856,"internally_reserved":0,"allocated":1511424,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.36799999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1507328,"data_stored":718931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.308}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.376},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.441},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.504}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.442},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.745},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.70999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.377}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.625},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:14:35.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:14:36.269 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:36.270 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:14:36.281 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-13T07:14:34.493967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16842,"ondisk_log_size":16842,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333148,"kb_used_data":11976,"kb_used_omap":20,"kb_used_meta":321003,"kb_avail":754641572,"statfs":{"total":773094113280,"available":772752969728,"internally_reserved":0,"allocated":12263424,"data_stored":6967138,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":328707328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070170"},"pg_stats":[{"pgid":"3.1f","version":"106'351","reported_seq":4067,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537193+0000","last_change":"2024-09-13T07:14:08.185500+0000","last_active":"2024-09-13T07:14:30.537193+0000","last_peered":"2024-09-13T07:14:30.537193+0000","last_clean":"2024-09-13T07:14:30.537193+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:30.537193+0000","last_undegraded":"2024-09-13T07:14:30.537193+0000","last_fullsized":"2024-09-13T07:14:30.537193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"106'377","reported_seq":4502,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506606+0000","last_change":"2024-09-13T07:14:08.147692+0000","last_active":"2024-09-13T07:14:29.506606+0000","last_peered":"2024-09-13T07:14:29.506606+0000","last_clean":"2024-09-13T07:14:29.506606+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:29.506606+0000","last_undegraded":"2024-09-13T07:14:29.506606+0000","last_fullsized":"2024-09-13T07:14:29.506606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"106'659","reported_seq":2765,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802835+0000","last_change":"2024-09-13T07:14:08.147661+0000","last_active":"2024-09-13T07:14:29.802835+0000","last_peered":"2024-09-13T07:14:29.802835+0000","last_clean":"2024-09-13T07:14:29.802835+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:29.802835+0000","last_undegraded":"2024-09-13T07:14:29.802835+0000","last_fullsized":"2024-09-13T07:14:29.802835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"106'508","reported_seq":3662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802799+0000","last_change":"2024-09-13T07:14:08.147649+0000","last_active":"2024-09-13T07:14:29.802799+0000","last_peered":"2024-09-13T07:14:29.802799+0000","last_clean":"2024-09-13T07:14:29.802799+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:29.802799+0000","last_undegraded":"2024-09-13T07:14:29.802799+0000","last_fullsized":"2024-09-13T07:14:29.802799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'1116","reported_seq":4756,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802255+0000","last_change":"2024-09-13T07:14:08.185626+0000","last_active":"2024-09-13T07:14:29.802255+0000","last_peered":"2024-09-13T07:14:29.802255+0000","last_clean":"2024-09-13T07:14:29.802255+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:29.802255+0000","last_undegraded":"2024-09-13T07:14:29.802255+0000","last_fullsized":"2024-09-13T07:14:29.802255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"101'357","reported_seq":4424,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803320+0000","last_change":"2024-09-13T07:14:08.153387+0000","last_active":"2024-09-13T07:14:29.803320+0000","last_peered":"2024-09-13T07:14:29.803320+0000","last_clean":"2024-09-13T07:14:29.803320+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:29.803320+0000","last_undegraded":"2024-09-13T07:14:29.803320+0000","last_fullsized":"2024-09-13T07:14:29.803320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"105'354","reported_seq":5316,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.431918+0000","last_change":"2024-09-13T07:14:08.255476+0000","last_active":"2024-09-13T07:14:29.431918+0000","last_peered":"2024-09-13T07:14:29.431918+0000","last_clean":"2024-09-13T07:14:29.431918+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:29.431918+0000","last_undegraded":"2024-09-13T07:14:29.431918+0000","last_fullsized":"2024-09-13T07:14:29.431918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"101'572","reported_seq":5592,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432239+0000","last_change":"2024-09-13T07:14:08.147872+0000","last_active":"2024-09-13T07:14:29.432239+0000","last_peered":"2024-09-13T07:14:29.432239+0000","last_clean":"2024-09-13T07:14:29.432239+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:29.432239+0000","last_undegraded":"2024-09-13T07:14:29.432239+0000","last_fullsized":"2024-09-13T07:14:29.432239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"106'499","reported_seq":3130,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802271+0000","last_change":"2024-09-13T07:14:08.185299+0000","last_active":"2024-09-13T07:14:29.802271+0000","last_peered":"2024-09-13T07:14:29.802271+0000","last_clean":"2024-09-13T07:14:29.802271+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:29.802271+0000","last_undegraded":"2024-09-13T07:14:29.802271+0000","last_fullsized":"2024-09-13T07:14:29.802271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"103'248","reported_seq":1373,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802691+0000","last_change":"2024-09-13T07:14:08.147385+0000","last_active":"2024-09-13T07:14:29.802691+0000","last_peered":"2024-09-13T07:14:29.802691+0000","last_clean":"2024-09-13T07:14:29.802691+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:29.802691+0000","last_undegraded":"2024-09-13T07:14:29.802691+0000","last_fullsized":"2024-09-13T07:14:29.802691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'337","reported_seq":4673,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802569+0000","last_change":"2024-09-13T07:14:08.151415+0000","last_active":"2024-09-13T07:14:29.802569+0000","last_peered":"2024-09-13T07:14:29.802569+0000","last_clean":"2024-09-13T07:14:29.802569+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:14:29.802569+0000","last_undegraded":"2024-09-13T07:14:29.802569+0000","last_fullsized":"2024-09-13T07:14:29.802569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"105'403","reported_seq":2120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803410+0000","last_change":"2024-09-13T07:14:08.153778+0000","last_active":"2024-09-13T07:14:29.803410+0000","last_peered":"2024-09-13T07:14:29.803410+0000","last_clean":"2024-09-13T07:14:29.803410+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:29.803410+0000","last_undegraded":"2024-09-13T07:14:29.803410+0000","last_fullsized":"2024-09-13T07:14:29.803410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438202+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:14:30.438202+0000","last_peered":"2024-09-13T07:14:30.438202+0000","last_clean":"2024-09-13T07:14:30.438202+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:30.438202+0000","last_undegraded":"2024-09-13T07:14:30.438202+0000","last_fullsized":"2024-09-13T07:14:30.438202+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"103'434","reported_seq":3836,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802922+0000","last_change":"2024-09-13T07:14:08.147440+0000","last_active":"2024-09-13T07:14:29.802922+0000","last_peered":"2024-09-13T07:14:29.802922+0000","last_clean":"2024-09-13T07:14:29.802922+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:29.802922+0000","last_undegraded":"2024-09-13T07:14:29.802922+0000","last_fullsized":"2024-09-13T07:14:29.802922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438237+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:14:30.438237+0000","last_peered":"2024-09-13T07:14:30.438237+0000","last_clean":"2024-09-13T07:14:30.438237+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:30.438237+0000","last_undegraded":"2024-09-13T07:14:30.438237+0000","last_fullsized":"2024-09-13T07:14:30.438237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"106'544","reported_seq":4371,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432122+0000","last_change":"2024-09-13T07:14:08.254223+0000","last_active":"2024-09-13T07:14:29.432122+0000","last_peered":"2024-09-13T07:14:29.432122+0000","last_clean":"2024-09-13T07:14:29.432122+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:29.432122+0000","last_undegraded":"2024-09-13T07:14:29.432122+0000","last_fullsized":"2024-09-13T07:14:29.432122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803432+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:14:29.803432+0000","last_peered":"2024-09-13T07:14:29.803432+0000","last_clean":"2024-09-13T07:14:29.803432+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:29.803432+0000","last_undegraded":"2024-09-13T07:14:29.803432+0000","last_fullsized":"2024-09-13T07:14:29.803432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"101'375","reported_seq":986,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432395+0000","last_change":"2024-09-13T07:14:08.148019+0000","last_active":"2024-09-13T07:14:29.432395+0000","last_peered":"2024-09-13T07:14:29.432395+0000","last_clean":"2024-09-13T07:14:29.432395+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:29.432395+0000","last_undegraded":"2024-09-13T07:14:29.432395+0000","last_fullsized":"2024-09-13T07:14:29.432395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506593+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:14:29.506593+0000","last_peered":"2024-09-13T07:14:29.506593+0000","last_clean":"2024-09-13T07:14:29.506593+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:29.506593+0000","last_undegraded":"2024-09-13T07:14:29.506593+0000","last_fullsized":"2024-09-13T07:14:29.506593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"106'386","reported_seq":7850,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802240+0000","last_change":"2024-09-13T07:14:08.185004+0000","last_active":"2024-09-13T07:14:29.802240+0000","last_peered":"2024-09-13T07:14:29.802240+0000","last_clean":"2024-09-13T07:14:29.802240+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:29.802240+0000","last_undegraded":"2024-09-13T07:14:29.802240+0000","last_fullsized":"2024-09-13T07:14:29.802240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"105'595","reported_seq":4780,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537058+0000","last_change":"2024-09-13T07:14:08.185870+0000","last_active":"2024-09-13T07:14:30.537058+0000","last_peered":"2024-09-13T07:14:30.537058+0000","last_clean":"2024-09-13T07:14:30.537058+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:30.537058+0000","last_undegraded":"2024-09-13T07:14:30.537058+0000","last_fullsized":"2024-09-13T07:14:30.537058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":243,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802501+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:14:29.802501+0000","last_peered":"2024-09-13T07:14:29.802501+0000","last_clean":"2024-09-13T07:14:29.802501+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:29.802501+0000","last_undegraded":"2024-09-13T07:14:29.802501+0000","last_fullsized":"2024-09-13T07:14:29.802501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":128,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503759+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:14:29.503759+0000","last_peered":"2024-09-13T07:14:29.503759+0000","last_clean":"2024-09-13T07:14:29.503759+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:29.503759+0000","last_undegraded":"2024-09-13T07:14:29.503759+0000","last_fullsized":"2024-09-13T07:14:29.503759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'467","reported_seq":2709,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503735+0000","last_change":"2024-09-13T07:14:08.255416+0000","last_active":"2024-09-13T07:14:29.503735+0000","last_peered":"2024-09-13T07:14:29.503735+0000","last_clean":"2024-09-13T07:14:29.503735+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:29.503735+0000","last_undegraded":"2024-09-13T07:14:29.503735+0000","last_fullsized":"2024-09-13T07:14:29.503735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802671+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:14:29.802671+0000","last_peered":"2024-09-13T07:14:29.802671+0000","last_clean":"2024-09-13T07:14:29.802671+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:14:29.802671+0000","last_undegraded":"2024-09-13T07:14:29.802671+0000","last_fullsized":"2024-09-13T07:14:29.802671+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"106'606","reported_seq":4415,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803401+0000","last_change":"2024-09-13T07:14:08.154052+0000","last_active":"2024-09-13T07:14:29.803401+0000","last_peered":"2024-09-13T07:14:29.803401+0000","last_clean":"2024-09-13T07:14:29.803401+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:29.803401+0000","last_undegraded":"2024-09-13T07:14:29.803401+0000","last_fullsized":"2024-09-13T07:14:29.803401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802669+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:14:29.802669+0000","last_peered":"2024-09-13T07:14:29.802669+0000","last_clean":"2024-09-13T07:14:29.802669+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:29.802669+0000","last_undegraded":"2024-09-13T07:14:29.802669+0000","last_fullsized":"2024-09-13T07:14:29.802669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"105'494","reported_seq":5282,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432302+0000","last_change":"2024-09-13T07:14:08.147309+0000","last_active":"2024-09-13T07:14:29.432302+0000","last_peered":"2024-09-13T07:14:29.432302+0000","last_clean":"2024-09-13T07:14:29.432302+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:29.432302+0000","last_undegraded":"2024-09-13T07:14:29.432302+0000","last_fullsized":"2024-09-13T07:14:29.432302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802464+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:14:29.802464+0000","last_peered":"2024-09-13T07:14:29.802464+0000","last_clean":"2024-09-13T07:14:29.802464+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:29.802464+0000","last_undegraded":"2024-09-13T07:14:29.802464+0000","last_fullsized":"2024-09-13T07:14:29.802464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'476","reported_seq":2374,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432192+0000","last_change":"2024-09-13T07:14:13.315962+0000","last_active":"2024-09-13T07:14:29.432192+0000","last_peered":"2024-09-13T07:14:29.432192+0000","last_clean":"2024-09-13T07:14:29.432192+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:29.432192+0000","last_undegraded":"2024-09-13T07:14:29.432192+0000","last_fullsized":"2024-09-13T07:14:29.432192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:39.120242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"104'520","reported_seq":1525,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803295+0000","last_change":"2024-09-13T07:14:08.153704+0000","last_active":"2024-09-13T07:14:29.803295+0000","last_peered":"2024-09-13T07:14:29.803295+0000","last_clean":"2024-09-13T07:14:29.803295+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:29.803295+0000","last_undegraded":"2024-09-13T07:14:29.803295+0000","last_fullsized":"2024-09-13T07:14:29.803295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"104'824","reported_seq":3683,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432245+0000","last_change":"2024-09-13T07:14:08.175514+0000","last_active":"2024-09-13T07:14:29.432245+0000","last_peered":"2024-09-13T07:14:29.432245+0000","last_clean":"2024-09-13T07:14:29.432245+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:29.432245+0000","last_undegraded":"2024-09-13T07:14:29.432245+0000","last_fullsized":"2024-09-13T07:14:29.432245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"106'861","reported_seq":3925,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432295+0000","last_change":"2024-09-13T07:14:08.147574+0000","last_active":"2024-09-13T07:14:29.432295+0000","last_peered":"2024-09-13T07:14:29.432295+0000","last_clean":"2024-09-13T07:14:29.432295+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:29.432295+0000","last_undegraded":"2024-09-13T07:14:29.432295+0000","last_fullsized":"2024-09-13T07:14:29.432295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'319","reported_seq":2404,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802698+0000","last_change":"2024-09-13T07:14:08.147077+0000","last_active":"2024-09-13T07:14:29.802698+0000","last_peered":"2024-09-13T07:14:29.802698+0000","last_clean":"2024-09-13T07:14:29.802698+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:29.802698+0000","last_undegraded":"2024-09-13T07:14:29.802698+0000","last_fullsized":"2024-09-13T07:14:29.802698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'379","reported_seq":2462,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802571+0000","last_change":"2024-09-13T07:14:08.151192+0000","last_active":"2024-09-13T07:14:29.802571+0000","last_peered":"2024-09-13T07:14:29.802571+0000","last_clean":"2024-09-13T07:14:29.802571+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:14:29.802571+0000","last_undegraded":"2024-09-13T07:14:29.802571+0000","last_fullsized":"2024-09-13T07:14:29.802571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"106'708","reported_seq":2117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802887+0000","last_change":"2024-09-13T07:14:08.151147+0000","last_active":"2024-09-13T07:14:29.802887+0000","last_peered":"2024-09-13T07:14:29.802887+0000","last_clean":"2024-09-13T07:14:29.802887+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:14:29.802887+0000","last_undegraded":"2024-09-13T07:14:29.802887+0000","last_fullsized":"2024-09-13T07:14:29.802887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"106'1061","reported_seq":7532,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802711+0000","last_change":"2024-09-13T07:14:08.151555+0000","last_active":"2024-09-13T07:14:29.802711+0000","last_peered":"2024-09-13T07:14:29.802711+0000","last_clean":"2024-09-13T07:14:29.802711+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:14:29.802711+0000","last_undegraded":"2024-09-13T07:14:29.802711+0000","last_fullsized":"2024-09-13T07:14:29.802711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"106'447","reported_seq":2522,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432275+0000","last_change":"2024-09-13T07:14:08.175489+0000","last_active":"2024-09-13T07:14:29.432275+0000","last_peered":"2024-09-13T07:14:29.432275+0000","last_clean":"2024-09-13T07:14:29.432275+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:29.432275+0000","last_undegraded":"2024-09-13T07:14:29.432275+0000","last_fullsized":"2024-09-13T07:14:29.432275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"104'464","reported_seq":3904,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802525+0000","last_change":"2024-09-13T07:14:08.150089+0000","last_active":"2024-09-13T07:14:29.802525+0000","last_peered":"2024-09-13T07:14:29.802525+0000","last_clean":"2024-09-13T07:14:29.802525+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:14:29.802525+0000","last_undegraded":"2024-09-13T07:14:29.802525+0000","last_fullsized":"2024-09-13T07:14:29.802525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"105'291","reported_seq":1892,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802205+0000","last_change":"2024-09-13T07:14:08.185172+0000","last_active":"2024-09-13T07:14:29.802205+0000","last_peered":"2024-09-13T07:14:29.802205+0000","last_clean":"2024-09-13T07:14:29.802205+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:29.802205+0000","last_undegraded":"2024-09-13T07:14:29.802205+0000","last_fullsized":"2024-09-13T07:14:29.802205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"103'690","reported_seq":4171,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802547+0000","last_change":"2024-09-13T07:14:08.150123+0000","last_active":"2024-09-13T07:14:29.802547+0000","last_peered":"2024-09-13T07:14:29.802547+0000","last_clean":"2024-09-13T07:14:29.802547+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:14:29.802547+0000","last_undegraded":"2024-09-13T07:14:29.802547+0000","last_fullsized":"2024-09-13T07:14:29.802547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":16722,"ondisk_log_size":16722,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266706,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38428,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94333412,"statfs":{"total":96636764160,"available":96597413888,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.63},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.5},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.436}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.80500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.751},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28512,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343328,"statfs":{"total":96636764160,"available":96607567872,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.433},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.34599999999999997}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.37}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64596,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307244,"statfs":{"total":96636764160,"available":96570617856,"internally_reserved":0,"allocated":1511424,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.36799999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1507328,"data_stored":718931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.308}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.376},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.441},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.504}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.442},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.745},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.70999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.377}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.625},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:14:36.284 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T07:14:36.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:14:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:36.574 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:14:36.584 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-13T07:14:36.494269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16842,"ondisk_log_size":16842,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333172,"kb_used_data":12000,"kb_used_omap":20,"kb_used_meta":321003,"kb_avail":754641548,"statfs":{"total":773094113280,"available":772752945152,"internally_reserved":0,"allocated":12288000,"data_stored":6978764,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":328707328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001626"},"pg_stats":[{"pgid":"3.1f","version":"106'351","reported_seq":4067,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537193+0000","last_change":"2024-09-13T07:14:08.185500+0000","last_active":"2024-09-13T07:14:30.537193+0000","last_peered":"2024-09-13T07:14:30.537193+0000","last_clean":"2024-09-13T07:14:30.537193+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:30.537193+0000","last_undegraded":"2024-09-13T07:14:30.537193+0000","last_fullsized":"2024-09-13T07:14:30.537193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:24.297250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"106'377","reported_seq":4502,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506606+0000","last_change":"2024-09-13T07:14:08.147692+0000","last_active":"2024-09-13T07:14:29.506606+0000","last_peered":"2024-09-13T07:14:29.506606+0000","last_clean":"2024-09-13T07:14:29.506606+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:29.506606+0000","last_undegraded":"2024-09-13T07:14:29.506606+0000","last_fullsized":"2024-09-13T07:14:29.506606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"106'659","reported_seq":2765,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802835+0000","last_change":"2024-09-13T07:14:08.147661+0000","last_active":"2024-09-13T07:14:29.802835+0000","last_peered":"2024-09-13T07:14:29.802835+0000","last_clean":"2024-09-13T07:14:29.802835+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:29.802835+0000","last_undegraded":"2024-09-13T07:14:29.802835+0000","last_fullsized":"2024-09-13T07:14:29.802835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"106'508","reported_seq":3662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802799+0000","last_change":"2024-09-13T07:14:08.147649+0000","last_active":"2024-09-13T07:14:29.802799+0000","last_peered":"2024-09-13T07:14:29.802799+0000","last_clean":"2024-09-13T07:14:29.802799+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:29.802799+0000","last_undegraded":"2024-09-13T07:14:29.802799+0000","last_fullsized":"2024-09-13T07:14:29.802799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'1116","reported_seq":4756,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802255+0000","last_change":"2024-09-13T07:14:08.185626+0000","last_active":"2024-09-13T07:14:29.802255+0000","last_peered":"2024-09-13T07:14:29.802255+0000","last_clean":"2024-09-13T07:14:29.802255+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:29.802255+0000","last_undegraded":"2024-09-13T07:14:29.802255+0000","last_fullsized":"2024-09-13T07:14:29.802255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:54.572728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"101'357","reported_seq":4424,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803320+0000","last_change":"2024-09-13T07:14:08.153387+0000","last_active":"2024-09-13T07:14:29.803320+0000","last_peered":"2024-09-13T07:14:29.803320+0000","last_clean":"2024-09-13T07:14:29.803320+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:29.803320+0000","last_undegraded":"2024-09-13T07:14:29.803320+0000","last_fullsized":"2024-09-13T07:14:29.803320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"105'354","reported_seq":5316,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.431918+0000","last_change":"2024-09-13T07:14:08.255476+0000","last_active":"2024-09-13T07:14:29.431918+0000","last_peered":"2024-09-13T07:14:29.431918+0000","last_clean":"2024-09-13T07:14:29.431918+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:29.431918+0000","last_undegraded":"2024-09-13T07:14:29.431918+0000","last_fullsized":"2024-09-13T07:14:29.431918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:35.908550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"101'572","reported_seq":5592,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432239+0000","last_change":"2024-09-13T07:14:08.147872+0000","last_active":"2024-09-13T07:14:29.432239+0000","last_peered":"2024-09-13T07:14:29.432239+0000","last_clean":"2024-09-13T07:14:29.432239+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:29.432239+0000","last_undegraded":"2024-09-13T07:14:29.432239+0000","last_fullsized":"2024-09-13T07:14:29.432239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"106'499","reported_seq":3130,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802271+0000","last_change":"2024-09-13T07:14:08.185299+0000","last_active":"2024-09-13T07:14:29.802271+0000","last_peered":"2024-09-13T07:14:29.802271+0000","last_clean":"2024-09-13T07:14:29.802271+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:29.802271+0000","last_undegraded":"2024-09-13T07:14:29.802271+0000","last_fullsized":"2024-09-13T07:14:29.802271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:26.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"103'248","reported_seq":1373,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802691+0000","last_change":"2024-09-13T07:14:08.147385+0000","last_active":"2024-09-13T07:14:29.802691+0000","last_peered":"2024-09-13T07:14:29.802691+0000","last_clean":"2024-09-13T07:14:29.802691+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:29.802691+0000","last_undegraded":"2024-09-13T07:14:29.802691+0000","last_fullsized":"2024-09-13T07:14:29.802691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'337","reported_seq":4673,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802569+0000","last_change":"2024-09-13T07:14:08.151415+0000","last_active":"2024-09-13T07:14:29.802569+0000","last_peered":"2024-09-13T07:14:29.802569+0000","last_clean":"2024-09-13T07:14:29.802569+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:14:29.802569+0000","last_undegraded":"2024-09-13T07:14:29.802569+0000","last_fullsized":"2024-09-13T07:14:29.802569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"105'403","reported_seq":2120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803410+0000","last_change":"2024-09-13T07:14:08.153778+0000","last_active":"2024-09-13T07:14:29.803410+0000","last_peered":"2024-09-13T07:14:29.803410+0000","last_clean":"2024-09-13T07:14:29.803410+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:29.803410+0000","last_undegraded":"2024-09-13T07:14:29.803410+0000","last_fullsized":"2024-09-13T07:14:29.803410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438202+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:14:30.438202+0000","last_peered":"2024-09-13T07:14:30.438202+0000","last_clean":"2024-09-13T07:14:30.438202+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:30.438202+0000","last_undegraded":"2024-09-13T07:14:30.438202+0000","last_fullsized":"2024-09-13T07:14:30.438202+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"103'434","reported_seq":3837,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537369+0000","last_change":"2024-09-13T07:14:08.147440+0000","last_active":"2024-09-13T07:14:30.537369+0000","last_peered":"2024-09-13T07:14:30.537369+0000","last_clean":"2024-09-13T07:14:30.537369+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:30.537369+0000","last_undegraded":"2024-09-13T07:14:30.537369+0000","last_fullsized":"2024-09-13T07:14:30.537369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438237+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:14:30.438237+0000","last_peered":"2024-09-13T07:14:30.438237+0000","last_clean":"2024-09-13T07:14:30.438237+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:30.438237+0000","last_undegraded":"2024-09-13T07:14:30.438237+0000","last_fullsized":"2024-09-13T07:14:30.438237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"106'544","reported_seq":4372,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.536808+0000","last_change":"2024-09-13T07:14:08.254223+0000","last_active":"2024-09-13T07:14:30.536808+0000","last_peered":"2024-09-13T07:14:30.536808+0000","last_clean":"2024-09-13T07:14:30.536808+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:30.536808+0000","last_undegraded":"2024-09-13T07:14:30.536808+0000","last_fullsized":"2024-09-13T07:14:30.536808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:07.040611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803432+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:14:29.803432+0000","last_peered":"2024-09-13T07:14:29.803432+0000","last_clean":"2024-09-13T07:14:29.803432+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:29.803432+0000","last_undegraded":"2024-09-13T07:14:29.803432+0000","last_fullsized":"2024-09-13T07:14:29.803432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"101'375","reported_seq":986,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432395+0000","last_change":"2024-09-13T07:14:08.148019+0000","last_active":"2024-09-13T07:14:29.432395+0000","last_peered":"2024-09-13T07:14:29.432395+0000","last_clean":"2024-09-13T07:14:29.432395+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:29.432395+0000","last_undegraded":"2024-09-13T07:14:29.432395+0000","last_fullsized":"2024-09-13T07:14:29.432395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506593+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:14:29.506593+0000","last_peered":"2024-09-13T07:14:29.506593+0000","last_clean":"2024-09-13T07:14:29.506593+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:29.506593+0000","last_undegraded":"2024-09-13T07:14:29.506593+0000","last_fullsized":"2024-09-13T07:14:29.506593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"106'386","reported_seq":7850,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802240+0000","last_change":"2024-09-13T07:14:08.185004+0000","last_active":"2024-09-13T07:14:29.802240+0000","last_peered":"2024-09-13T07:14:29.802240+0000","last_clean":"2024-09-13T07:14:29.802240+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:29.802240+0000","last_undegraded":"2024-09-13T07:14:29.802240+0000","last_fullsized":"2024-09-13T07:14:29.802240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:15:49.319756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"105'595","reported_seq":4780,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537058+0000","last_change":"2024-09-13T07:14:08.185870+0000","last_active":"2024-09-13T07:14:30.537058+0000","last_peered":"2024-09-13T07:14:30.537058+0000","last_clean":"2024-09-13T07:14:30.537058+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:30.537058+0000","last_undegraded":"2024-09-13T07:14:30.537058+0000","last_fullsized":"2024-09-13T07:14:30.537058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:50.278801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":243,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802501+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:14:29.802501+0000","last_peered":"2024-09-13T07:14:29.802501+0000","last_clean":"2024-09-13T07:14:29.802501+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:29.802501+0000","last_undegraded":"2024-09-13T07:14:29.802501+0000","last_fullsized":"2024-09-13T07:14:29.802501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":128,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503759+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:14:29.503759+0000","last_peered":"2024-09-13T07:14:29.503759+0000","last_clean":"2024-09-13T07:14:29.503759+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:29.503759+0000","last_undegraded":"2024-09-13T07:14:29.503759+0000","last_fullsized":"2024-09-13T07:14:29.503759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:07.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'467","reported_seq":2709,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.503735+0000","last_change":"2024-09-13T07:14:08.255416+0000","last_active":"2024-09-13T07:14:29.503735+0000","last_peered":"2024-09-13T07:14:29.503735+0000","last_clean":"2024-09-13T07:14:29.503735+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:29.503735+0000","last_undegraded":"2024-09-13T07:14:29.503735+0000","last_fullsized":"2024-09-13T07:14:29.503735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:23.042111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802671+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:14:29.802671+0000","last_peered":"2024-09-13T07:14:29.802671+0000","last_clean":"2024-09-13T07:14:29.802671+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:14:29.802671+0000","last_undegraded":"2024-09-13T07:14:29.802671+0000","last_fullsized":"2024-09-13T07:14:29.802671+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"106'606","reported_seq":4416,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.516460+0000","last_change":"2024-09-13T07:14:08.154052+0000","last_active":"2024-09-13T07:14:30.516460+0000","last_peered":"2024-09-13T07:14:30.516460+0000","last_clean":"2024-09-13T07:14:30.516460+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:30.516460+0000","last_undegraded":"2024-09-13T07:14:30.516460+0000","last_fullsized":"2024-09-13T07:14:30.516460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802669+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:14:29.802669+0000","last_peered":"2024-09-13T07:14:29.802669+0000","last_clean":"2024-09-13T07:14:29.802669+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:29.802669+0000","last_undegraded":"2024-09-13T07:14:29.802669+0000","last_fullsized":"2024-09-13T07:14:29.802669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"105'494","reported_seq":5283,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.441090+0000","last_change":"2024-09-13T07:14:08.147309+0000","last_active":"2024-09-13T07:14:30.441090+0000","last_peered":"2024-09-13T07:14:30.441090+0000","last_clean":"2024-09-13T07:14:30.441090+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:30.441090+0000","last_undegraded":"2024-09-13T07:14:30.441090+0000","last_fullsized":"2024-09-13T07:14:30.441090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802464+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:14:29.802464+0000","last_peered":"2024-09-13T07:14:29.802464+0000","last_clean":"2024-09-13T07:14:29.802464+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:29.802464+0000","last_undegraded":"2024-09-13T07:14:29.802464+0000","last_fullsized":"2024-09-13T07:14:29.802464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'476","reported_seq":2374,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432192+0000","last_change":"2024-09-13T07:14:13.315962+0000","last_active":"2024-09-13T07:14:29.432192+0000","last_peered":"2024-09-13T07:14:29.432192+0000","last_clean":"2024-09-13T07:14:29.432192+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:29.432192+0000","last_undegraded":"2024-09-13T07:14:29.432192+0000","last_fullsized":"2024-09-13T07:14:29.432192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:39.120242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"104'520","reported_seq":1525,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803295+0000","last_change":"2024-09-13T07:14:08.153704+0000","last_active":"2024-09-13T07:14:29.803295+0000","last_peered":"2024-09-13T07:14:29.803295+0000","last_clean":"2024-09-13T07:14:29.803295+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:29.803295+0000","last_undegraded":"2024-09-13T07:14:29.803295+0000","last_fullsized":"2024-09-13T07:14:29.803295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"104'824","reported_seq":3683,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432245+0000","last_change":"2024-09-13T07:14:08.175514+0000","last_active":"2024-09-13T07:14:29.432245+0000","last_peered":"2024-09-13T07:14:29.432245+0000","last_clean":"2024-09-13T07:14:29.432245+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:29.432245+0000","last_undegraded":"2024-09-13T07:14:29.432245+0000","last_fullsized":"2024-09-13T07:14:29.432245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"106'861","reported_seq":3925,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432295+0000","last_change":"2024-09-13T07:14:08.147574+0000","last_active":"2024-09-13T07:14:29.432295+0000","last_peered":"2024-09-13T07:14:29.432295+0000","last_clean":"2024-09-13T07:14:29.432295+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:29.432295+0000","last_undegraded":"2024-09-13T07:14:29.432295+0000","last_fullsized":"2024-09-13T07:14:29.432295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'319","reported_seq":2404,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802698+0000","last_change":"2024-09-13T07:14:08.147077+0000","last_active":"2024-09-13T07:14:29.802698+0000","last_peered":"2024-09-13T07:14:29.802698+0000","last_clean":"2024-09-13T07:14:29.802698+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:29.802698+0000","last_undegraded":"2024-09-13T07:14:29.802698+0000","last_fullsized":"2024-09-13T07:14:29.802698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'379","reported_seq":2462,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802571+0000","last_change":"2024-09-13T07:14:08.151192+0000","last_active":"2024-09-13T07:14:29.802571+0000","last_peered":"2024-09-13T07:14:29.802571+0000","last_clean":"2024-09-13T07:14:29.802571+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:14:29.802571+0000","last_undegraded":"2024-09-13T07:14:29.802571+0000","last_fullsized":"2024-09-13T07:14:29.802571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"106'708","reported_seq":2117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802887+0000","last_change":"2024-09-13T07:14:08.151147+0000","last_active":"2024-09-13T07:14:29.802887+0000","last_peered":"2024-09-13T07:14:29.802887+0000","last_clean":"2024-09-13T07:14:29.802887+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:14:29.802887+0000","last_undegraded":"2024-09-13T07:14:29.802887+0000","last_fullsized":"2024-09-13T07:14:29.802887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"106'1061","reported_seq":7532,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802711+0000","last_change":"2024-09-13T07:14:08.151555+0000","last_active":"2024-09-13T07:14:29.802711+0000","last_peered":"2024-09-13T07:14:29.802711+0000","last_clean":"2024-09-13T07:14:29.802711+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:14:29.802711+0000","last_undegraded":"2024-09-13T07:14:29.802711+0000","last_fullsized":"2024-09-13T07:14:29.802711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"106'447","reported_seq":2522,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432275+0000","last_change":"2024-09-13T07:14:08.175489+0000","last_active":"2024-09-13T07:14:29.432275+0000","last_peered":"2024-09-13T07:14:29.432275+0000","last_clean":"2024-09-13T07:14:29.432275+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:29.432275+0000","last_undegraded":"2024-09-13T07:14:29.432275+0000","last_fullsized":"2024-09-13T07:14:29.432275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"104'464","reported_seq":3904,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802525+0000","last_change":"2024-09-13T07:14:08.150089+0000","last_active":"2024-09-13T07:14:29.802525+0000","last_peered":"2024-09-13T07:14:29.802525+0000","last_clean":"2024-09-13T07:14:29.802525+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:14:29.802525+0000","last_undegraded":"2024-09-13T07:14:29.802525+0000","last_fullsized":"2024-09-13T07:14:29.802525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"105'291","reported_seq":1892,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802205+0000","last_change":"2024-09-13T07:14:08.185172+0000","last_active":"2024-09-13T07:14:29.802205+0000","last_peered":"2024-09-13T07:14:29.802205+0000","last_clean":"2024-09-13T07:14:29.802205+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:29.802205+0000","last_undegraded":"2024-09-13T07:14:29.802205+0000","last_fullsized":"2024-09-13T07:14:29.802205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:47.836364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"103'690","reported_seq":4171,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802547+0000","last_change":"2024-09-13T07:14:08.150123+0000","last_active":"2024-09-13T07:14:29.802547+0000","last_peered":"2024-09-13T07:14:29.802547+0000","last_clean":"2024-09-13T07:14:29.802547+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:14:29.802547+0000","last_undegraded":"2024-09-13T07:14:29.802547+0000","last_fullsized":"2024-09-13T07:14:29.802547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":16722,"ondisk_log_size":16722,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266706,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38428,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94333412,"statfs":{"total":96636764160,"available":96597413888,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.63},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.5},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.436}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.80500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28512,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343328,"statfs":{"total":96636764160,"available":96607567872,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.63},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.504},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.57799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64604,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307236,"statfs":{"total":96636764160,"available":96570609664,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65800000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.623},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.64400000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.503},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.378}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.504}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56299999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.75}]},{"osd":7,"last update":"Fri Sep 13 07:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.35199999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45400000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.40799999999999997}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:14:36.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:14:36.873 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:36.873 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"f667c631-bb9d-47bf-97c6-7217d8893244","created":"2024-09-13T07:12:08.346514+0000","modified":"2024-09-13T07:14:30.434428+0000","last_up_change":"2024-09-13T07:13:13.343075+0000","last_in_change":"2024-09-13T07:12:10.626165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T07:12:19.268005+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T07:12:20.513150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T07:12:28.796583+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":73,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c29fb342-d45b-48c1-9694-dd01e2a01325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6809","nonce":1709347595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6811","nonce":1709347595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6815","nonce":1709347595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1709347595},{"type":"v1","addr":"172.21.15.31:6813","nonce":1709347595}]},"public_addr":"172.21.15.31:6809/1709347595","cluster_addr":"172.21.15.31:6811/1709347595","heartbeat_back_addr":"172.21.15.31:6815/1709347595","heartbeat_front_addr":"172.21.15.31:6813/1709347595","state":["exists","up"]},{"osd":1,"uuid":"4b979b5c-a78b-461d-b795-f6f5c25cf652","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6825","nonce":3300827638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6827","nonce":3300827638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6831","nonce":3300827638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3300827638},{"type":"v1","addr":"172.21.15.31:6829","nonce":3300827638}]},"public_addr":"172.21.15.31:6825/3300827638","cluster_addr":"172.21.15.31:6827/3300827638","heartbeat_back_addr":"172.21.15.31:6831/3300827638","heartbeat_front_addr":"172.21.15.31:6829/3300827638","state":["exists","up"]},{"osd":2,"uuid":"11259f54-1134-495f-9281-d5132aff2780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6801","nonce":3977233096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6803","nonce":3977233096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6807","nonce":3977233096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3977233096},{"type":"v1","addr":"172.21.15.31:6805","nonce":3977233096}]},"public_addr":"172.21.15.31:6801/3977233096","cluster_addr":"172.21.15.31:6803/3977233096","heartbeat_back_addr":"172.21.15.31:6807/3977233096","heartbeat_front_addr":"172.21.15.31:6805/3977233096","state":["exists","up"]},{"osd":3,"uuid":"a11be2be-3803-4a23-bbf8-40ea110ccf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6817","nonce":3290545828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6819","nonce":3290545828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6823","nonce":3290545828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3290545828},{"type":"v1","addr":"172.21.15.31:6821","nonce":3290545828}]},"public_addr":"172.21.15.31:6817/3290545828","cluster_addr":"172.21.15.31:6819/3290545828","heartbeat_back_addr":"172.21.15.31:6823/3290545828","heartbeat_front_addr":"172.21.15.31:6821/3290545828","state":["exists","up"]},{"osd":4,"uuid":"2b6941d9-29f5-4836-a61c-47ff27a7172e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6801","nonce":3543694628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6803","nonce":3543694628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6807","nonce":3543694628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3543694628},{"type":"v1","addr":"172.21.15.184:6805","nonce":3543694628}]},"public_addr":"172.21.15.184:6801/3543694628","cluster_addr":"172.21.15.184:6803/3543694628","heartbeat_back_addr":"172.21.15.184:6807/3543694628","heartbeat_front_addr":"172.21.15.184:6805/3543694628","state":["exists","up"]},{"osd":5,"uuid":"17e3a02d-28e3-4126-a8f2-c7783e524d62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6810","nonce":951246042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6814","nonce":951246042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6821","nonce":951246042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":951246042},{"type":"v1","addr":"172.21.15.184:6818","nonce":951246042}]},"public_addr":"172.21.15.184:6810/951246042","cluster_addr":"172.21.15.184:6814/951246042","heartbeat_back_addr":"172.21.15.184:6821/951246042","heartbeat_front_addr":"172.21.15.184:6818/951246042","state":["exists","up"]},{"osd":6,"uuid":"ddf29f57-7733-42fa-94ed-015a529fb689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6825","nonce":1941674146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6827","nonce":1941674146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6831","nonce":1941674146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1941674146},{"type":"v1","addr":"172.21.15.184:6829","nonce":1941674146}]},"public_addr":"172.21.15.184:6825/1941674146","cluster_addr":"172.21.15.184:6827/1941674146","heartbeat_back_addr":"172.21.15.184:6831/1941674146","heartbeat_front_addr":"172.21.15.184:6829/1941674146","state":["exists","up"]},{"osd":7,"uuid":"d91e2eed-3bfb-4eb0-ac30-18770c106742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":30,"up_from":53,"up_thru":56,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6811","nonce":48890479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6815","nonce":48890479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6823","nonce":48890479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6817","nonce":48890479},{"type":"v1","addr":"172.21.15.184:6819","nonce":48890479}]},"public_addr":"172.21.15.184:6811/48890479","cluster_addr":"172.21.15.184:6815/48890479","heartbeat_back_addr":"172.21.15.184:6823/48890479","heartbeat_front_addr":"172.21.15.184:6819/48890479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.805494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.817444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.712083+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.739958+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.720767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.715944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.788853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:12:15.809325+0000","dead_epoch":31}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:14:37.886 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T07:14:37.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:38.025 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:38.025 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:38.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T07:14:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T07:14:38.333 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T07:14:38.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:38.459 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:38.459 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:38.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T07:14:38.755 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T07:14:38.768 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T07:14:38.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:38.894 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:38.894 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:38.894 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:38.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T07:14:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T07:14:39.201 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T07:14:39.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:39.326 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:39.327 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:39.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T07:14:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T07:14:39.640 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T07:14:39.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:39.769 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:39.770 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:39.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T07:14:40.066 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T07:14:40.079 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T07:14:40.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:40.205 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:40.206 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:40.206 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:40.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T07:14:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T07:14:40.514 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T07:14:40.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:40.639 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:40.639 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:40.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T07:14:40.942 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T07:14:40.955 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T07:14:40.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:14:41.081 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T07:14:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:14:41.081 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T07:14:41.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T07:14:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T07:14:41.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:14:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:14:41.675 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:14:41.686 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-13T07:14:40.495001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16842,"ondisk_log_size":16842,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333172,"kb_used_data":12000,"kb_used_omap":20,"kb_used_meta":321003,"kb_avail":754641548,"statfs":{"total":773094113280,"available":772752945152,"internally_reserved":0,"allocated":12288000,"data_stored":6978764,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":328707328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.058170"},"pg_stats":[{"pgid":"3.1f","version":"106'351","reported_seq":4068,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:31.284582+0000","last_change":"2024-09-13T07:14:08.185500+0000","last_active":"2024-09-13T07:14:31.284582+0000","last_peered":"2024-09-13T07:14:31.284582+0000","last_clean":"2024-09-13T07:14:31.284582+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:31.284582+0000","last_undegraded":"2024-09-13T07:14:31.284582+0000","last_fullsized":"2024-09-13T07:14:31.284582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"106'377","reported_seq":4502,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506606+0000","last_change":"2024-09-13T07:14:08.147692+0000","last_active":"2024-09-13T07:14:29.506606+0000","last_peered":"2024-09-13T07:14:29.506606+0000","last_clean":"2024-09-13T07:14:29.506606+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:29.506606+0000","last_undegraded":"2024-09-13T07:14:29.506606+0000","last_fullsized":"2024-09-13T07:14:29.506606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:37.078192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"106'659","reported_seq":2765,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802835+0000","last_change":"2024-09-13T07:14:08.147661+0000","last_active":"2024-09-13T07:14:29.802835+0000","last_peered":"2024-09-13T07:14:29.802835+0000","last_clean":"2024-09-13T07:14:29.802835+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:29.802835+0000","last_undegraded":"2024-09-13T07:14:29.802835+0000","last_fullsized":"2024-09-13T07:14:29.802835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:55.492960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"106'508","reported_seq":3662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802799+0000","last_change":"2024-09-13T07:14:08.147649+0000","last_active":"2024-09-13T07:14:29.802799+0000","last_peered":"2024-09-13T07:14:29.802799+0000","last_clean":"2024-09-13T07:14:29.802799+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:29.802799+0000","last_undegraded":"2024-09-13T07:14:29.802799+0000","last_fullsized":"2024-09-13T07:14:29.802799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:12:14.544322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'1116","reported_seq":4757,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537206+0000","last_change":"2024-09-13T07:14:08.185626+0000","last_active":"2024-09-13T07:14:30.537206+0000","last_peered":"2024-09-13T07:14:30.537206+0000","last_clean":"2024-09-13T07:14:30.537206+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:30.537206+0000","last_undegraded":"2024-09-13T07:14:30.537206+0000","last_fullsized":"2024-09-13T07:14:30.537206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"101'357","reported_seq":4424,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803320+0000","last_change":"2024-09-13T07:14:08.153387+0000","last_active":"2024-09-13T07:14:29.803320+0000","last_peered":"2024-09-13T07:14:29.803320+0000","last_clean":"2024-09-13T07:14:29.803320+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:29.803320+0000","last_undegraded":"2024-09-13T07:14:29.803320+0000","last_fullsized":"2024-09-13T07:14:29.803320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:27.624334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"105'354","reported_seq":5317,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.536872+0000","last_change":"2024-09-13T07:14:08.255476+0000","last_active":"2024-09-13T07:14:30.536872+0000","last_peered":"2024-09-13T07:14:30.536872+0000","last_clean":"2024-09-13T07:14:30.536872+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:30.536872+0000","last_undegraded":"2024-09-13T07:14:30.536872+0000","last_fullsized":"2024-09-13T07:14:30.536872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"101'572","reported_seq":5592,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432239+0000","last_change":"2024-09-13T07:14:08.147872+0000","last_active":"2024-09-13T07:14:29.432239+0000","last_peered":"2024-09-13T07:14:29.432239+0000","last_clean":"2024-09-13T07:14:29.432239+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:29.432239+0000","last_undegraded":"2024-09-13T07:14:29.432239+0000","last_fullsized":"2024-09-13T07:14:29.432239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:45.344672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"106'499","reported_seq":3131,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537224+0000","last_change":"2024-09-13T07:14:08.185299+0000","last_active":"2024-09-13T07:14:30.537224+0000","last_peered":"2024-09-13T07:14:30.537224+0000","last_clean":"2024-09-13T07:14:30.537224+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:30.537224+0000","last_undegraded":"2024-09-13T07:14:30.537224+0000","last_fullsized":"2024-09-13T07:14:30.537224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"103'248","reported_seq":1373,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802691+0000","last_change":"2024-09-13T07:14:08.147385+0000","last_active":"2024-09-13T07:14:29.802691+0000","last_peered":"2024-09-13T07:14:29.802691+0000","last_clean":"2024-09-13T07:14:29.802691+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:29.802691+0000","last_undegraded":"2024-09-13T07:14:29.802691+0000","last_fullsized":"2024-09-13T07:14:29.802691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:57.448750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'337","reported_seq":4673,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802569+0000","last_change":"2024-09-13T07:14:08.151415+0000","last_active":"2024-09-13T07:14:29.802569+0000","last_peered":"2024-09-13T07:14:29.802569+0000","last_clean":"2024-09-13T07:14:29.802569+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:14:29.802569+0000","last_undegraded":"2024-09-13T07:14:29.802569+0000","last_fullsized":"2024-09-13T07:14:29.802569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:54:00.913005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"105'403","reported_seq":2120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803410+0000","last_change":"2024-09-13T07:14:08.153778+0000","last_active":"2024-09-13T07:14:29.803410+0000","last_peered":"2024-09-13T07:14:29.803410+0000","last_clean":"2024-09-13T07:14:29.803410+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:29.803410+0000","last_undegraded":"2024-09-13T07:14:29.803410+0000","last_fullsized":"2024-09-13T07:14:29.803410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:28.838641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438202+0000","last_change":"2024-09-13T07:12:22.590999+0000","last_active":"2024-09-13T07:14:30.438202+0000","last_peered":"2024-09-13T07:14:30.438202+0000","last_clean":"2024-09-13T07:14:30.438202+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:30.438202+0000","last_undegraded":"2024-09-13T07:14:30.438202+0000","last_fullsized":"2024-09-13T07:14:30.438202+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:11.538890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"103'434","reported_seq":3837,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537369+0000","last_change":"2024-09-13T07:14:08.147440+0000","last_active":"2024-09-13T07:14:30.537369+0000","last_peered":"2024-09-13T07:14:30.537369+0000","last_clean":"2024-09-13T07:14:30.537369+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:30.537369+0000","last_undegraded":"2024-09-13T07:14:30.537369+0000","last_fullsized":"2024-09-13T07:14:30.537369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:17.052894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438237+0000","last_change":"2024-09-13T07:12:22.590994+0000","last_active":"2024-09-13T07:14:30.438237+0000","last_peered":"2024-09-13T07:14:30.438237+0000","last_clean":"2024-09-13T07:14:30.438237+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:30.438237+0000","last_undegraded":"2024-09-13T07:14:30.438237+0000","last_fullsized":"2024-09-13T07:14:30.438237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:17.464133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"106'544","reported_seq":4373,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:34.504146+0000","last_change":"2024-09-13T07:14:08.254223+0000","last_active":"2024-09-13T07:14:34.504146+0000","last_peered":"2024-09-13T07:14:34.504146+0000","last_clean":"2024-09-13T07:14:34.504146+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:34.504146+0000","last_undegraded":"2024-09-13T07:14:34.504146+0000","last_fullsized":"2024-09-13T07:14:34.504146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.803432+0000","last_change":"2024-09-13T07:12:22.863101+0000","last_active":"2024-09-13T07:14:29.803432+0000","last_peered":"2024-09-13T07:14:29.803432+0000","last_clean":"2024-09-13T07:14:29.803432+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:29.803432+0000","last_undegraded":"2024-09-13T07:14:29.803432+0000","last_fullsized":"2024-09-13T07:14:29.803432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:17.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"101'375","reported_seq":987,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.441068+0000","last_change":"2024-09-13T07:14:08.148019+0000","last_active":"2024-09-13T07:14:30.441068+0000","last_peered":"2024-09-13T07:14:30.441068+0000","last_clean":"2024-09-13T07:14:30.441068+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:30.441068+0000","last_undegraded":"2024-09-13T07:14:30.441068+0000","last_fullsized":"2024-09-13T07:14:30.441068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:56.432961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.506593+0000","last_change":"2024-09-13T07:12:22.592265+0000","last_active":"2024-09-13T07:14:29.506593+0000","last_peered":"2024-09-13T07:14:29.506593+0000","last_clean":"2024-09-13T07:14:29.506593+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:29.506593+0000","last_undegraded":"2024-09-13T07:14:29.506593+0000","last_fullsized":"2024-09-13T07:14:29.506593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:15:55.822365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"106'386","reported_seq":7852,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:38.322743+0000","last_change":"2024-09-13T07:14:08.185004+0000","last_active":"2024-09-13T07:14:38.322743+0000","last_peered":"2024-09-13T07:14:38.322743+0000","last_clean":"2024-09-13T07:14:38.322743+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:38.322743+0000","last_undegraded":"2024-09-13T07:14:38.322743+0000","last_fullsized":"2024-09-13T07:14:38.322743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"105'595","reported_seq":4781,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:31.284516+0000","last_change":"2024-09-13T07:14:08.185870+0000","last_active":"2024-09-13T07:14:31.284516+0000","last_peered":"2024-09-13T07:14:31.284516+0000","last_clean":"2024-09-13T07:14:31.284516+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:31.284516+0000","last_undegraded":"2024-09-13T07:14:31.284516+0000","last_fullsized":"2024-09-13T07:14:31.284516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":243,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802501+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:14:29.802501+0000","last_peered":"2024-09-13T07:14:29.802501+0000","last_clean":"2024-09-13T07:14:29.802501+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:29.802501+0000","last_undegraded":"2024-09-13T07:14:29.802501+0000","last_fullsized":"2024-09-13T07:14:29.802501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:04.657416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":129,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.536991+0000","last_change":"2024-09-13T07:12:22.592051+0000","last_active":"2024-09-13T07:14:30.536991+0000","last_peered":"2024-09-13T07:14:30.536991+0000","last_clean":"2024-09-13T07:14:30.536991+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:30.536991+0000","last_undegraded":"2024-09-13T07:14:30.536991+0000","last_fullsized":"2024-09-13T07:14:30.536991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'467","reported_seq":2710,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.536808+0000","last_change":"2024-09-13T07:14:08.255416+0000","last_active":"2024-09-13T07:14:30.536808+0000","last_peered":"2024-09-13T07:14:30.536808+0000","last_clean":"2024-09-13T07:14:30.536808+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:30.536808+0000","last_undegraded":"2024-09-13T07:14:30.536808+0000","last_fullsized":"2024-09-13T07:14:30.536808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802671+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:14:29.802671+0000","last_peered":"2024-09-13T07:14:29.802671+0000","last_clean":"2024-09-13T07:14:29.802671+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:14:29.802671+0000","last_undegraded":"2024-09-13T07:14:29.802671+0000","last_fullsized":"2024-09-13T07:14:29.802671+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.303538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"106'606","reported_seq":4416,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.516460+0000","last_change":"2024-09-13T07:14:08.154052+0000","last_active":"2024-09-13T07:14:30.516460+0000","last_peered":"2024-09-13T07:14:30.516460+0000","last_clean":"2024-09-13T07:14:30.516460+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:30.516460+0000","last_undegraded":"2024-09-13T07:14:30.516460+0000","last_fullsized":"2024-09-13T07:14:30.516460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:11.899857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802669+0000","last_change":"2024-09-13T07:12:22.742180+0000","last_active":"2024-09-13T07:14:29.802669+0000","last_peered":"2024-09-13T07:14:29.802669+0000","last_clean":"2024-09-13T07:14:29.802669+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:29.802669+0000","last_undegraded":"2024-09-13T07:14:29.802669+0000","last_fullsized":"2024-09-13T07:14:29.802669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:44.923910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"105'494","reported_seq":5283,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.441090+0000","last_change":"2024-09-13T07:14:08.147309+0000","last_active":"2024-09-13T07:14:30.441090+0000","last_peered":"2024-09-13T07:14:30.441090+0000","last_clean":"2024-09-13T07:14:30.441090+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:30.441090+0000","last_undegraded":"2024-09-13T07:14:30.441090+0000","last_fullsized":"2024-09-13T07:14:30.441090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:26.247376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802464+0000","last_change":"2024-09-13T07:13:14.353725+0000","last_active":"2024-09-13T07:14:29.802464+0000","last_peered":"2024-09-13T07:14:29.802464+0000","last_clean":"2024-09-13T07:14:29.802464+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:29.802464+0000","last_undegraded":"2024-09-13T07:14:29.802464+0000","last_fullsized":"2024-09-13T07:14:29.802464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:20:05.783472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'476","reported_seq":2374,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432192+0000","last_change":"2024-09-13T07:14:13.315962+0000","last_active":"2024-09-13T07:14:29.432192+0000","last_peered":"2024-09-13T07:14:29.432192+0000","last_clean":"2024-09-13T07:14:29.432192+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:29.432192+0000","last_undegraded":"2024-09-13T07:14:29.432192+0000","last_fullsized":"2024-09-13T07:14:29.432192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:39.120242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"104'520","reported_seq":1526,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.516259+0000","last_change":"2024-09-13T07:14:08.153704+0000","last_active":"2024-09-13T07:14:30.516259+0000","last_peered":"2024-09-13T07:14:30.516259+0000","last_clean":"2024-09-13T07:14:30.516259+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:30.516259+0000","last_undegraded":"2024-09-13T07:14:30.516259+0000","last_fullsized":"2024-09-13T07:14:30.516259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:13.915061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"104'824","reported_seq":3684,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.438236+0000","last_change":"2024-09-13T07:14:08.175514+0000","last_active":"2024-09-13T07:14:30.438236+0000","last_peered":"2024-09-13T07:14:30.438236+0000","last_clean":"2024-09-13T07:14:30.438236+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:30.438236+0000","last_undegraded":"2024-09-13T07:14:30.438236+0000","last_fullsized":"2024-09-13T07:14:30.438236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:22:04.850843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"106'861","reported_seq":3925,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432295+0000","last_change":"2024-09-13T07:14:08.147574+0000","last_active":"2024-09-13T07:14:29.432295+0000","last_peered":"2024-09-13T07:14:29.432295+0000","last_clean":"2024-09-13T07:14:29.432295+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:29.432295+0000","last_undegraded":"2024-09-13T07:14:29.432295+0000","last_fullsized":"2024-09-13T07:14:29.432295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:20:24.635516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'319","reported_seq":2404,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802698+0000","last_change":"2024-09-13T07:14:08.147077+0000","last_active":"2024-09-13T07:14:29.802698+0000","last_peered":"2024-09-13T07:14:29.802698+0000","last_clean":"2024-09-13T07:14:29.802698+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:29.802698+0000","last_undegraded":"2024-09-13T07:14:29.802698+0000","last_fullsized":"2024-09-13T07:14:29.802698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:57:05.311035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'379","reported_seq":2462,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802571+0000","last_change":"2024-09-13T07:14:08.151192+0000","last_active":"2024-09-13T07:14:29.802571+0000","last_peered":"2024-09-13T07:14:29.802571+0000","last_clean":"2024-09-13T07:14:29.802571+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:14:29.802571+0000","last_undegraded":"2024-09-13T07:14:29.802571+0000","last_fullsized":"2024-09-13T07:14:29.802571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:46.522303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"106'708","reported_seq":2117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802887+0000","last_change":"2024-09-13T07:14:08.151147+0000","last_active":"2024-09-13T07:14:29.802887+0000","last_peered":"2024-09-13T07:14:29.802887+0000","last_clean":"2024-09-13T07:14:29.802887+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:14:29.802887+0000","last_undegraded":"2024-09-13T07:14:29.802887+0000","last_fullsized":"2024-09-13T07:14:29.802887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:31.838231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"106'1061","reported_seq":7532,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802711+0000","last_change":"2024-09-13T07:14:08.151555+0000","last_active":"2024-09-13T07:14:29.802711+0000","last_peered":"2024-09-13T07:14:29.802711+0000","last_clean":"2024-09-13T07:14:29.802711+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:14:29.802711+0000","last_undegraded":"2024-09-13T07:14:29.802711+0000","last_fullsized":"2024-09-13T07:14:29.802711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:26.337129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"106'447","reported_seq":2522,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.432275+0000","last_change":"2024-09-13T07:14:08.175489+0000","last_active":"2024-09-13T07:14:29.432275+0000","last_peered":"2024-09-13T07:14:29.432275+0000","last_clean":"2024-09-13T07:14:29.432275+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:29.432275+0000","last_undegraded":"2024-09-13T07:14:29.432275+0000","last_fullsized":"2024-09-13T07:14:29.432275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:52:49.713033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"104'464","reported_seq":3904,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802525+0000","last_change":"2024-09-13T07:14:08.150089+0000","last_active":"2024-09-13T07:14:29.802525+0000","last_peered":"2024-09-13T07:14:29.802525+0000","last_clean":"2024-09-13T07:14:29.802525+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:14:29.802525+0000","last_undegraded":"2024-09-13T07:14:29.802525+0000","last_fullsized":"2024-09-13T07:14:29.802525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:31:10.717226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"105'291","reported_seq":1893,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537217+0000","last_change":"2024-09-13T07:14:08.185172+0000","last_active":"2024-09-13T07:14:30.537217+0000","last_peered":"2024-09-13T07:14:30.537217+0000","last_clean":"2024-09-13T07:14:30.537217+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:30.537217+0000","last_undegraded":"2024-09-13T07:14:30.537217+0000","last_fullsized":"2024-09-13T07:14:30.537217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"103'690","reported_seq":4171,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-13T07:14:29.802547+0000","last_change":"2024-09-13T07:14:08.150123+0000","last_active":"2024-09-13T07:14:29.802547+0000","last_peered":"2024-09-13T07:14:29.802547+0000","last_clean":"2024-09-13T07:14:29.802547+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:14:29.802547+0000","last_undegraded":"2024-09-13T07:14:29.802547+0000","last_fullsized":"2024-09-13T07:14:29.802547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:46.468217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":16722,"ondisk_log_size":16722,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266707,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38428,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":36854,"kb_avail":94333412,"statfs":{"total":96636764160,"available":96597413888,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":37738643},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.442}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.63800000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.625}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.66600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40100000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.503}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28512,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343328,"statfs":{"total":96636764160,"available":96607567872,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.498},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.629}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.441},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64604,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307236,"statfs":{"total":96636764160,"available":96570609664,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.71199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.255}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35899999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.23599999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.499}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.77100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.75}]},{"osd":7,"last update":"Fri Sep 13 07:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.75700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:14:41.692 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.692 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.692 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.692 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.693 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.694 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T07:12:20.574171+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=20, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.695 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:14:41.696 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T07:15:01.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:15:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:15:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:15:02.012 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-13T07:15:00.499633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17237,"ondisk_log_size":17237,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337184,"kb_used_data":12000,"kb_used_omap":20,"kb_used_meta":325099,"kb_avail":754637536,"statfs":{"total":773094113280,"available":772748836864,"internally_reserved":0,"allocated":12288000,"data_stored":6978764,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":332901632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002939"},"pg_stats":[{"pgid":"3.1f","version":"127'368","reported_seq":4096,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:50.811994+0000","last_change":"2024-09-13T07:14:50.812111+0000","last_active":"2024-09-13T07:14:50.811994+0000","last_peered":"2024-09-13T07:14:50.811994+0000","last_clean":"2024-09-13T07:14:50.811994+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:50.811994+0000","last_undegraded":"2024-09-13T07:14:50.811994+0000","last_fullsized":"2024-09-13T07:14:50.811994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'368","last_scrub_stamp":"2024-09-13T07:14:50.811965+0000","last_deep_scrub":"127'368","last_deep_scrub_stamp":"2024-09-13T07:14:50.811965+0000","last_clean_scrub_stamp":"2024-09-13T07:14:50.811965+0000","objects_scrubbed":17,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:27:49.658021+0000","scrub_duration":0.010490943000000001,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"127'407","reported_seq":4546,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:46.569748+0000","last_change":"2024-09-13T07:14:46.569849+0000","last_active":"2024-09-13T07:14:46.569748+0000","last_peered":"2024-09-13T07:14:46.569748+0000","last_clean":"2024-09-13T07:14:46.569748+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:46.569748+0000","last_undegraded":"2024-09-13T07:14:46.569748+0000","last_fullsized":"2024-09-13T07:14:46.569748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'407","last_scrub_stamp":"2024-09-13T07:14:46.569727+0000","last_deep_scrub":"127'407","last_deep_scrub_stamp":"2024-09-13T07:14:46.569727+0000","last_clean_scrub_stamp":"2024-09-13T07:14:46.569727+0000","objects_scrubbed":30,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:45:46.366206+0000","scrub_duration":0.022967922000000002,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"106'659","reported_seq":2767,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:40.502867+0000","last_change":"2024-09-13T07:14:08.147661+0000","last_active":"2024-09-13T07:14:40.502867+0000","last_peered":"2024-09-13T07:14:40.502867+0000","last_clean":"2024-09-13T07:14:40.502867+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:40.502867+0000","last_undegraded":"2024-09-13T07:14:40.502867+0000","last_fullsized":"2024-09-13T07:14:40.502867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"127'526","reported_seq":3693,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:46.747863+0000","last_change":"2024-09-13T07:14:46.747863+0000","last_active":"2024-09-13T07:14:46.747863+0000","last_peered":"2024-09-13T07:14:46.747863+0000","last_clean":"2024-09-13T07:14:46.747863+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:46.747863+0000","last_undegraded":"2024-09-13T07:14:46.747863+0000","last_fullsized":"2024-09-13T07:14:46.747863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'526","last_scrub_stamp":"2024-09-13T07:14:46.747706+0000","last_deep_scrub":"127'526","last_deep_scrub_stamp":"2024-09-13T07:14:46.747706+0000","last_clean_scrub_stamp":"2024-09-13T07:14:46.747706+0000","objects_scrubbed":18,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:29:48.192131+0000","scrub_duration":0.01319126,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"127'1136","reported_seq":4789,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:43.880407+0000","last_change":"2024-09-13T07:14:43.880407+0000","last_active":"2024-09-13T07:14:43.880407+0000","last_peered":"2024-09-13T07:14:43.880407+0000","last_clean":"2024-09-13T07:14:43.880407+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:43.880407+0000","last_undegraded":"2024-09-13T07:14:43.880407+0000","last_fullsized":"2024-09-13T07:14:43.880407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'1136","last_scrub_stamp":"2024-09-13T07:14:43.880219+0000","last_deep_scrub":"127'1136","last_deep_scrub_stamp":"2024-09-13T07:14:43.880219+0000","last_clean_scrub_stamp":"2024-09-13T07:14:43.880219+0000","objects_scrubbed":20,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:40.036943+0000","scrub_duration":0.017069734999999999,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"127'372","reported_seq":4452,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:45.514893+0000","last_change":"2024-09-13T07:14:45.514893+0000","last_active":"2024-09-13T07:14:45.514893+0000","last_peered":"2024-09-13T07:14:45.514893+0000","last_clean":"2024-09-13T07:14:45.514893+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:45.514893+0000","last_undegraded":"2024-09-13T07:14:45.514893+0000","last_fullsized":"2024-09-13T07:14:45.514893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'372","last_scrub_stamp":"2024-09-13T07:14:45.514673+0000","last_deep_scrub":"127'372","last_deep_scrub_stamp":"2024-09-13T07:14:45.514673+0000","last_clean_scrub_stamp":"2024-09-13T07:14:45.514673+0000","objects_scrubbed":15,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:48.312032+0000","scrub_duration":0.013833412999999999,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"127'368","reported_seq":5343,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:41.673532+0000","last_change":"2024-09-13T07:14:41.673532+0000","last_active":"2024-09-13T07:14:41.673532+0000","last_peered":"2024-09-13T07:14:41.673532+0000","last_clean":"2024-09-13T07:14:41.673532+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:41.673532+0000","last_undegraded":"2024-09-13T07:14:41.673532+0000","last_fullsized":"2024-09-13T07:14:41.673532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'368","last_scrub_stamp":"2024-09-13T07:14:41.673414+0000","last_deep_scrub":"127'368","last_deep_scrub_stamp":"2024-09-13T07:14:41.673414+0000","last_clean_scrub_stamp":"2024-09-13T07:14:41.673414+0000","objects_scrubbed":14,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:11:22.900037+0000","scrub_duration":0.0088143490000000008,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"127'595","reported_seq":5628,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:48.515146+0000","last_change":"2024-09-13T07:14:48.515146+0000","last_active":"2024-09-13T07:14:48.515146+0000","last_peered":"2024-09-13T07:14:48.515146+0000","last_clean":"2024-09-13T07:14:48.515146+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:48.515146+0000","last_undegraded":"2024-09-13T07:14:48.515146+0000","last_fullsized":"2024-09-13T07:14:48.515146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'595","last_scrub_stamp":"2024-09-13T07:14:48.514960+0000","last_deep_scrub":"127'595","last_deep_scrub_stamp":"2024-09-13T07:14:48.514960+0000","last_clean_scrub_stamp":"2024-09-13T07:14:48.514960+0000","objects_scrubbed":23,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:33:47.773675+0000","scrub_duration":0.017592561999999999,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"127'510","reported_seq":3154,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:40.890351+0000","last_change":"2024-09-13T07:14:40.890351+0000","last_active":"2024-09-13T07:14:40.890351+0000","last_peered":"2024-09-13T07:14:40.890351+0000","last_clean":"2024-09-13T07:14:40.890351+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:40.890351+0000","last_undegraded":"2024-09-13T07:14:40.890351+0000","last_fullsized":"2024-09-13T07:14:40.890351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'510","last_scrub_stamp":"2024-09-13T07:14:40.890238+0000","last_deep_scrub":"127'510","last_deep_scrub_stamp":"2024-09-13T07:14:40.890238+0000","last_clean_scrub_stamp":"2024-09-13T07:14:40.890238+0000","objects_scrubbed":11,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:48.017258+0000","scrub_duration":0.012146716,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"127'269","reported_seq":1407,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:52.747971+0000","last_change":"2024-09-13T07:14:52.747971+0000","last_active":"2024-09-13T07:14:52.747971+0000","last_peered":"2024-09-13T07:14:52.747971+0000","last_clean":"2024-09-13T07:14:52.747971+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:52.747971+0000","last_undegraded":"2024-09-13T07:14:52.747971+0000","last_fullsized":"2024-09-13T07:14:52.747971+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'269","last_scrub_stamp":"2024-09-13T07:14:52.747805+0000","last_deep_scrub":"127'269","last_deep_scrub_stamp":"2024-09-13T07:14:52.747805+0000","last_clean_scrub_stamp":"2024-09-13T07:14:52.747805+0000","objects_scrubbed":21,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:59.165280+0000","scrub_duration":0.015694957999999998,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'337","reported_seq":4674,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537128+0000","last_change":"2024-09-13T07:14:08.151415+0000","last_active":"2024-09-13T07:14:30.537128+0000","last_peered":"2024-09-13T07:14:30.537128+0000","last_clean":"2024-09-13T07:14:30.537128+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:14:30.537128+0000","last_undegraded":"2024-09-13T07:14:30.537128+0000","last_fullsized":"2024-09-13T07:14:30.537128+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"127'408","reported_seq":2138,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:49.578230+0000","last_change":"2024-09-13T07:14:49.578387+0000","last_active":"2024-09-13T07:14:49.578230+0000","last_peered":"2024-09-13T07:14:49.578230+0000","last_clean":"2024-09-13T07:14:49.578230+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:49.578230+0000","last_undegraded":"2024-09-13T07:14:49.578230+0000","last_fullsized":"2024-09-13T07:14:49.578230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'408","last_scrub_stamp":"2024-09-13T07:14:49.578192+0000","last_deep_scrub":"127'408","last_deep_scrub_stamp":"2024-09-13T07:14:49.578192+0000","last_clean_scrub_stamp":"2024-09-13T07:14:49.578192+0000","objects_scrubbed":5,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:11.168960+0000","scrub_duration":0.0096498039999999997,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":136,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.275696+0000","last_change":"2024-09-13T07:14:42.275862+0000","last_active":"2024-09-13T07:14:42.275696+0000","last_peered":"2024-09-13T07:14:42.275696+0000","last_clean":"2024-09-13T07:14:42.275696+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:42.275696+0000","last_undegraded":"2024-09-13T07:14:42.275696+0000","last_fullsized":"2024-09-13T07:14:42.275696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:42.275675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:42.275675+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.275675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:33:04.922579+0000","scrub_duration":0.0018022450000000001,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"127'450","reported_seq":3865,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:50.731334+0000","last_change":"2024-09-13T07:14:50.731453+0000","last_active":"2024-09-13T07:14:50.731334+0000","last_peered":"2024-09-13T07:14:50.731334+0000","last_clean":"2024-09-13T07:14:50.731334+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:50.731334+0000","last_undegraded":"2024-09-13T07:14:50.731334+0000","last_fullsized":"2024-09-13T07:14:50.731334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'450","last_scrub_stamp":"2024-09-13T07:14:50.731308+0000","last_deep_scrub":"127'450","last_deep_scrub_stamp":"2024-09-13T07:14:50.731308+0000","last_clean_scrub_stamp":"2024-09-13T07:14:50.731308+0000","objects_scrubbed":16,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:26.717623+0000","scrub_duration":0.010798248,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":136,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:41.301543+0000","last_change":"2024-09-13T07:14:41.301645+0000","last_active":"2024-09-13T07:14:41.301543+0000","last_peered":"2024-09-13T07:14:41.301543+0000","last_clean":"2024-09-13T07:14:41.301543+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:41.301543+0000","last_undegraded":"2024-09-13T07:14:41.301543+0000","last_fullsized":"2024-09-13T07:14:41.301543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:41.301521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:41.301521+0000","last_clean_scrub_stamp":"2024-09-13T07:14:41.301521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:54.722336+0000","scrub_duration":0.001478416,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"127'556","reported_seq":4396,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:44.649207+0000","last_change":"2024-09-13T07:14:44.649318+0000","last_active":"2024-09-13T07:14:44.649207+0000","last_peered":"2024-09-13T07:14:44.649207+0000","last_clean":"2024-09-13T07:14:44.649207+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:44.649207+0000","last_undegraded":"2024-09-13T07:14:44.649207+0000","last_fullsized":"2024-09-13T07:14:44.649207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'556","last_scrub_stamp":"2024-09-13T07:14:44.649184+0000","last_deep_scrub":"127'556","last_deep_scrub_stamp":"2024-09-13T07:14:44.649184+0000","last_clean_scrub_stamp":"2024-09-13T07:14:44.649184+0000","objects_scrubbed":12,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:38.078340+0000","scrub_duration":0.010280350000000001,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":135,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:43.514132+0000","last_change":"2024-09-13T07:14:43.514273+0000","last_active":"2024-09-13T07:14:43.514132+0000","last_peered":"2024-09-13T07:14:43.514132+0000","last_clean":"2024-09-13T07:14:43.514132+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:43.514132+0000","last_undegraded":"2024-09-13T07:14:43.514132+0000","last_fullsized":"2024-09-13T07:14:43.514132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:43.514110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:43.514110+0000","last_clean_scrub_stamp":"2024-09-13T07:14:43.514110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:17:56.064096+0000","scrub_duration":0.0016024590000000001,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"127'393","reported_seq":1017,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:45.519244+0000","last_change":"2024-09-13T07:14:45.519359+0000","last_active":"2024-09-13T07:14:45.519244+0000","last_peered":"2024-09-13T07:14:45.519244+0000","last_clean":"2024-09-13T07:14:45.519244+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:45.519244+0000","last_undegraded":"2024-09-13T07:14:45.519244+0000","last_fullsized":"2024-09-13T07:14:45.519244+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'393","last_scrub_stamp":"2024-09-13T07:14:45.519221+0000","last_deep_scrub":"127'393","last_deep_scrub_stamp":"2024-09-13T07:14:45.519221+0000","last_clean_scrub_stamp":"2024-09-13T07:14:45.519221+0000","objects_scrubbed":18,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:33.482832+0000","scrub_duration":0.0124332,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":138,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:40.446065+0000","last_change":"2024-09-13T07:14:40.446156+0000","last_active":"2024-09-13T07:14:40.446065+0000","last_peered":"2024-09-13T07:14:40.446065+0000","last_clean":"2024-09-13T07:14:40.446065+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:40.446065+0000","last_undegraded":"2024-09-13T07:14:40.446065+0000","last_fullsized":"2024-09-13T07:14:40.446065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:40.446047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:40.446047+0000","last_clean_scrub_stamp":"2024-09-13T07:14:40.446047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:54.143634+0000","scrub_duration":0.0018531439999999999,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"127'398","reported_seq":7875,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.885662+0000","last_change":"2024-09-13T07:14:42.885836+0000","last_active":"2024-09-13T07:14:42.885662+0000","last_peered":"2024-09-13T07:14:42.885662+0000","last_clean":"2024-09-13T07:14:42.885662+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:42.885662+0000","last_undegraded":"2024-09-13T07:14:42.885662+0000","last_fullsized":"2024-09-13T07:14:42.885662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'398","last_scrub_stamp":"2024-09-13T07:14:42.885638+0000","last_deep_scrub":"127'398","last_deep_scrub_stamp":"2024-09-13T07:14:42.885638+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.885638+0000","objects_scrubbed":12,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:33.170148+0000","scrub_duration":0.012168109999999999,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"127'608","reported_seq":4805,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:39.872464+0000","last_change":"2024-09-13T07:14:39.872567+0000","last_active":"2024-09-13T07:14:39.872464+0000","last_peered":"2024-09-13T07:14:39.872464+0000","last_clean":"2024-09-13T07:14:39.872464+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:39.872464+0000","last_undegraded":"2024-09-13T07:14:39.872464+0000","last_fullsized":"2024-09-13T07:14:39.872464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'608","last_scrub_stamp":"2024-09-13T07:14:39.872444+0000","last_deep_scrub":"127'608","last_deep_scrub_stamp":"2024-09-13T07:14:39.872444+0000","last_clean_scrub_stamp":"2024-09-13T07:14:39.872444+0000","objects_scrubbed":13,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:41.830143+0000","scrub_duration":0.0088363620000000004,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":244,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.536965+0000","last_change":"2024-09-13T07:13:14.355634+0000","last_active":"2024-09-13T07:14:30.536965+0000","last_peered":"2024-09-13T07:14:30.536965+0000","last_clean":"2024-09-13T07:14:30.536965+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:30.536965+0000","last_undegraded":"2024-09-13T07:14:30.536965+0000","last_fullsized":"2024-09-13T07:14:30.536965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:20.574171+0000","last_clean_scrub_stamp":"2024-09-13T07:12:20.574171+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"127'2","reported_seq":141,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:39.700458+0000","last_change":"2024-09-13T07:14:39.700576+0000","last_active":"2024-09-13T07:14:39.700458+0000","last_peered":"2024-09-13T07:14:39.700458+0000","last_clean":"2024-09-13T07:14:39.700458+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:39.700458+0000","last_undegraded":"2024-09-13T07:14:39.700458+0000","last_fullsized":"2024-09-13T07:14:39.700458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'2","last_scrub_stamp":"2024-09-13T07:14:39.700438+0000","last_deep_scrub":"127'2","last_deep_scrub_stamp":"2024-09-13T07:14:39.700438+0000","last_clean_scrub_stamp":"2024-09-13T07:14:39.700438+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:46.103029+0000","scrub_duration":0.003116237,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"127'478","reported_seq":2732,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:53.585170+0000","last_change":"2024-09-13T07:14:53.585302+0000","last_active":"2024-09-13T07:14:53.585170+0000","last_peered":"2024-09-13T07:14:53.585170+0000","last_clean":"2024-09-13T07:14:53.585170+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:53.585170+0000","last_undegraded":"2024-09-13T07:14:53.585170+0000","last_fullsized":"2024-09-13T07:14:53.585170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'478","last_scrub_stamp":"2024-09-13T07:14:53.585144+0000","last_deep_scrub":"127'478","last_deep_scrub_stamp":"2024-09-13T07:14:53.585144+0000","last_clean_scrub_stamp":"2024-09-13T07:14:53.585144+0000","objects_scrubbed":11,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:38.289471+0000","scrub_duration":0.0086389510000000006,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":105,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537101+0000","last_change":"2024-09-13T07:13:14.354321+0000","last_active":"2024-09-13T07:14:30.537101+0000","last_peered":"2024-09-13T07:14:30.537101+0000","last_clean":"2024-09-13T07:14:30.537101+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:14:30.537101+0000","last_undegraded":"2024-09-13T07:14:30.537101+0000","last_fullsized":"2024-09-13T07:14:30.537101+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:19.568703+0000","last_clean_scrub_stamp":"2024-09-13T07:12:19.568703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"127'629","reported_seq":4451,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:48.605163+0000","last_change":"2024-09-13T07:14:48.605282+0000","last_active":"2024-09-13T07:14:48.605163+0000","last_peered":"2024-09-13T07:14:48.605163+0000","last_clean":"2024-09-13T07:14:48.605163+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:48.605163+0000","last_undegraded":"2024-09-13T07:14:48.605163+0000","last_fullsized":"2024-09-13T07:14:48.605163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'629","last_scrub_stamp":"2024-09-13T07:14:48.605137+0000","last_deep_scrub":"127'629","last_deep_scrub_stamp":"2024-09-13T07:14:48.605137+0000","last_clean_scrub_stamp":"2024-09-13T07:14:48.605137+0000","objects_scrubbed":23,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:13.839766+0000","scrub_duration":0.017911283,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":137,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.777247+0000","last_change":"2024-09-13T07:14:42.777404+0000","last_active":"2024-09-13T07:14:42.777247+0000","last_peered":"2024-09-13T07:14:42.777247+0000","last_clean":"2024-09-13T07:14:42.777247+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:42.777247+0000","last_undegraded":"2024-09-13T07:14:42.777247+0000","last_fullsized":"2024-09-13T07:14:42.777247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:42.777224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:42.777224+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.777224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:19:03.168524+0000","scrub_duration":0.00202528,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"127'507","reported_seq":5308,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:47.542067+0000","last_change":"2024-09-13T07:14:47.542189+0000","last_active":"2024-09-13T07:14:47.542067+0000","last_peered":"2024-09-13T07:14:47.542067+0000","last_clean":"2024-09-13T07:14:47.542067+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:47.542067+0000","last_undegraded":"2024-09-13T07:14:47.542067+0000","last_fullsized":"2024-09-13T07:14:47.542067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'507","last_scrub_stamp":"2024-09-13T07:14:47.542044+0000","last_deep_scrub":"127'507","last_deep_scrub_stamp":"2024-09-13T07:14:47.542044+0000","last_clean_scrub_stamp":"2024-09-13T07:14:47.542044+0000","objects_scrubbed":13,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:32.829209+0000","scrub_duration":0.010914961000000001,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:51.673202+0000","last_change":"2024-09-13T07:14:51.673305+0000","last_active":"2024-09-13T07:14:51.673202+0000","last_peered":"2024-09-13T07:14:51.673202+0000","last_clean":"2024-09-13T07:14:51.673202+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:51.673202+0000","last_undegraded":"2024-09-13T07:14:51.673202+0000","last_fullsized":"2024-09-13T07:14:51.673202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:51.673180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:51.673180+0000","last_clean_scrub_stamp":"2024-09-13T07:14:51.673180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:12:10.822873+0000","scrub_duration":0.0022204659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"127'489","reported_seq":2399,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:51.450943+0000","last_change":"2024-09-13T07:14:51.451077+0000","last_active":"2024-09-13T07:14:51.450943+0000","last_peered":"2024-09-13T07:14:51.450943+0000","last_clean":"2024-09-13T07:14:51.450943+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:51.450943+0000","last_undegraded":"2024-09-13T07:14:51.450943+0000","last_fullsized":"2024-09-13T07:14:51.450943+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'489","last_scrub_stamp":"2024-09-13T07:14:51.450918+0000","last_deep_scrub":"127'489","last_deep_scrub_stamp":"2024-09-13T07:14:51.450918+0000","last_clean_scrub_stamp":"2024-09-13T07:14:51.450918+0000","objects_scrubbed":13,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:36.411832+0000","scrub_duration":0.010254340000000001,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"127'532","reported_seq":1550,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:54.609682+0000","last_change":"2024-09-13T07:14:54.609798+0000","last_active":"2024-09-13T07:14:54.609682+0000","last_peered":"2024-09-13T07:14:54.609682+0000","last_clean":"2024-09-13T07:14:54.609682+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:54.609682+0000","last_undegraded":"2024-09-13T07:14:54.609682+0000","last_fullsized":"2024-09-13T07:14:54.609682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'532","last_scrub_stamp":"2024-09-13T07:14:54.609659+0000","last_deep_scrub":"127'532","last_deep_scrub_stamp":"2024-09-13T07:14:54.609659+0000","last_clean_scrub_stamp":"2024-09-13T07:14:54.609659+0000","objects_scrubbed":12,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:42.461520+0000","scrub_duration":0.0088661380000000008,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"127'841","reported_seq":3713,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:44.246190+0000","last_change":"2024-09-13T07:14:44.246315+0000","last_active":"2024-09-13T07:14:44.246190+0000","last_peered":"2024-09-13T07:14:44.246190+0000","last_clean":"2024-09-13T07:14:44.246190+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:44.246190+0000","last_undegraded":"2024-09-13T07:14:44.246190+0000","last_fullsized":"2024-09-13T07:14:44.246190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'841","last_scrub_stamp":"2024-09-13T07:14:44.246165+0000","last_deep_scrub":"127'841","last_deep_scrub_stamp":"2024-09-13T07:14:44.246165+0000","last_clean_scrub_stamp":"2024-09-13T07:14:44.246165+0000","objects_scrubbed":17,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:13.210054+0000","scrub_duration":0.013817772000000001,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"127'870","reported_seq":3947,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.418304+0000","last_change":"2024-09-13T07:14:42.418304+0000","last_active":"2024-09-13T07:14:42.418304+0000","last_peered":"2024-09-13T07:14:42.418304+0000","last_clean":"2024-09-13T07:14:42.418304+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:42.418304+0000","last_undegraded":"2024-09-13T07:14:42.418304+0000","last_fullsized":"2024-09-13T07:14:42.418304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'870","last_scrub_stamp":"2024-09-13T07:14:42.418117+0000","last_deep_scrub":"127'870","last_deep_scrub_stamp":"2024-09-13T07:14:42.418117+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.418117+0000","objects_scrubbed":9,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:15.770471+0000","scrub_duration":0.0088902559999999992,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"127'331","reported_seq":2429,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:54.750111+0000","last_change":"2024-09-13T07:14:54.750234+0000","last_active":"2024-09-13T07:14:54.750111+0000","last_peered":"2024-09-13T07:14:54.750111+0000","last_clean":"2024-09-13T07:14:54.750111+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:54.750111+0000","last_undegraded":"2024-09-13T07:14:54.750111+0000","last_fullsized":"2024-09-13T07:14:54.750111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'331","last_scrub_stamp":"2024-09-13T07:14:54.750088+0000","last_deep_scrub":"127'331","last_deep_scrub_stamp":"2024-09-13T07:14:54.750088+0000","last_clean_scrub_stamp":"2024-09-13T07:14:54.750088+0000","objects_scrubbed":12,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:58.081201+0000","scrub_duration":0.0086890779999999994,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'379","reported_seq":2463,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537198+0000","last_change":"2024-09-13T07:14:08.151192+0000","last_active":"2024-09-13T07:14:30.537198+0000","last_peered":"2024-09-13T07:14:30.537198+0000","last_clean":"2024-09-13T07:14:30.537198+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:14:30.537198+0000","last_undegraded":"2024-09-13T07:14:30.537198+0000","last_fullsized":"2024-09-13T07:14:30.537198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"106'708","reported_seq":2118,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537261+0000","last_change":"2024-09-13T07:14:08.151147+0000","last_active":"2024-09-13T07:14:30.537261+0000","last_peered":"2024-09-13T07:14:30.537261+0000","last_clean":"2024-09-13T07:14:30.537261+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:14:30.537261+0000","last_undegraded":"2024-09-13T07:14:30.537261+0000","last_fullsized":"2024-09-13T07:14:30.537261+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"106'1061","reported_seq":7533,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537162+0000","last_change":"2024-09-13T07:14:08.151555+0000","last_active":"2024-09-13T07:14:30.537162+0000","last_peered":"2024-09-13T07:14:30.537162+0000","last_clean":"2024-09-13T07:14:30.537162+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:14:30.537162+0000","last_undegraded":"2024-09-13T07:14:30.537162+0000","last_fullsized":"2024-09-13T07:14:30.537162+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"127'465","reported_seq":2553,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:45.207709+0000","last_change":"2024-09-13T07:14:45.207709+0000","last_active":"2024-09-13T07:14:45.207709+0000","last_peered":"2024-09-13T07:14:45.207709+0000","last_clean":"2024-09-13T07:14:45.207709+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:45.207709+0000","last_undegraded":"2024-09-13T07:14:45.207709+0000","last_fullsized":"2024-09-13T07:14:45.207709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'465","last_scrub_stamp":"2024-09-13T07:14:45.207570+0000","last_deep_scrub":"127'465","last_deep_scrub_stamp":"2024-09-13T07:14:45.207570+0000","last_clean_scrub_stamp":"2024-09-13T07:14:45.207570+0000","objects_scrubbed":18,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:59.376286+0000","scrub_duration":0.016652568,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"104'464","reported_seq":3905,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.536991+0000","last_change":"2024-09-13T07:14:08.150089+0000","last_active":"2024-09-13T07:14:30.536991+0000","last_peered":"2024-09-13T07:14:30.536991+0000","last_clean":"2024-09-13T07:14:30.536991+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:14:30.536991+0000","last_undegraded":"2024-09-13T07:14:30.536991+0000","last_fullsized":"2024-09-13T07:14:30.536991+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"127'312","reported_seq":1926,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:44.861899+0000","last_change":"2024-09-13T07:14:44.861899+0000","last_active":"2024-09-13T07:14:44.861899+0000","last_peered":"2024-09-13T07:14:44.861899+0000","last_clean":"2024-09-13T07:14:44.861899+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:44.861899+0000","last_undegraded":"2024-09-13T07:14:44.861899+0000","last_fullsized":"2024-09-13T07:14:44.861899+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'312","last_scrub_stamp":"2024-09-13T07:14:44.861723+0000","last_deep_scrub":"127'312","last_deep_scrub_stamp":"2024-09-13T07:14:44.861723+0000","last_clean_scrub_stamp":"2024-09-13T07:14:44.861723+0000","objects_scrubbed":21,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:23.458200+0000","scrub_duration":0.013567737,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"103'690","reported_seq":4172,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:30.537101+0000","last_change":"2024-09-13T07:14:08.150123+0000","last_active":"2024-09-13T07:14:30.537101+0000","last_peered":"2024-09-13T07:14:30.537101+0000","last_clean":"2024-09-13T07:14:30.537101+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:14:30.537101+0000","last_undegraded":"2024-09-13T07:14:30.537101+0000","last_fullsized":"2024-09-13T07:14:30.537101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:12:29.128481+0000","last_clean_scrub_stamp":"2024-09-13T07:12:29.128481+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":17116,"ondisk_log_size":17116,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266711,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42512,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":40950,"kb_avail":94329328,"statfs":{"total":96636764160,"available":96593231872,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":41932947},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.624}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59499999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.372}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.41799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.623}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41799999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.44500000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.497}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.754},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.60799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64600,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307240,"statfs":{"total":96636764160,"available":96570613760,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.503},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.309},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.36799999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Sep 13 07:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.43099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.376},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.752},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40848,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330992,"statfs":{"total":96636764160,"available":96594935808,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.309},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.42799999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.624},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54744,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317096,"statfs":{"total":96636764160,"available":96580706304,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.374}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.498},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:15:02.015 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.015 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.016 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T07:12:20.574171+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=20, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.016 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-13T07:12:19.568703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.016 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.016 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.016 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.016 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.017 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T07:12:29.128481+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=12, tm_sec=29, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=14, tm_sec=36, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:15:02.017 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T07:15:22.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:15:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T07:15:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T07:15:22.331 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-13T07:15:20.504166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":501,"num_read":148145,"num_read_kb":627746,"num_write":20075,"num_write_kb":358994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17351,"ondisk_log_size":17351,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337184,"kb_used_data":12000,"kb_used_omap":20,"kb_used_meta":325099,"kb_avail":754637536,"statfs":{"total":773094113280,"available":772748836864,"internally_reserved":0,"allocated":12288000,"data_stored":6978764,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":21248,"internal_metadata":332901632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002544"},"pg_stats":[{"pgid":"3.1f","version":"127'368","reported_seq":4096,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:50.811994+0000","last_change":"2024-09-13T07:14:50.812111+0000","last_active":"2024-09-13T07:14:50.811994+0000","last_peered":"2024-09-13T07:14:50.811994+0000","last_clean":"2024-09-13T07:14:50.811994+0000","last_became_active":"2024-09-13T07:12:30.145582+0000","last_became_peered":"2024-09-13T07:12:30.145582+0000","last_unstale":"2024-09-13T07:14:50.811994+0000","last_undegraded":"2024-09-13T07:14:50.811994+0000","last_fullsized":"2024-09-13T07:14:50.811994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'368","last_scrub_stamp":"2024-09-13T07:14:50.811965+0000","last_deep_scrub":"127'368","last_deep_scrub_stamp":"2024-09-13T07:14:50.811965+0000","last_clean_scrub_stamp":"2024-09-13T07:14:50.811965+0000","objects_scrubbed":17,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:27:49.658021+0000","scrub_duration":0.010490943000000001,"objects_trimmed":0,"snaptrim_duration":0.00088456600000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3985,"num_read_kb":17488,"num_write":356,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"127'407","reported_seq":4546,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:46.569748+0000","last_change":"2024-09-13T07:14:46.569849+0000","last_active":"2024-09-13T07:14:46.569748+0000","last_peered":"2024-09-13T07:14:46.569748+0000","last_clean":"2024-09-13T07:14:46.569748+0000","last_became_active":"2024-09-13T07:12:30.145906+0000","last_became_peered":"2024-09-13T07:12:30.145906+0000","last_unstale":"2024-09-13T07:14:46.569748+0000","last_undegraded":"2024-09-13T07:14:46.569748+0000","last_fullsized":"2024-09-13T07:14:46.569748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'407","last_scrub_stamp":"2024-09-13T07:14:46.569727+0000","last_deep_scrub":"127'407","last_deep_scrub_stamp":"2024-09-13T07:14:46.569727+0000","last_clean_scrub_stamp":"2024-09-13T07:14:46.569727+0000","objects_scrubbed":30,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:45:46.366206+0000","scrub_duration":0.022967922000000002,"objects_trimmed":0,"snaptrim_duration":0.0009883209999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4797,"num_read_kb":20411,"num_write":306,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"127'677","reported_seq":2796,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:59.746078+0000","last_change":"2024-09-13T07:14:59.746239+0000","last_active":"2024-09-13T07:14:59.746078+0000","last_peered":"2024-09-13T07:14:59.746078+0000","last_clean":"2024-09-13T07:14:59.746078+0000","last_became_active":"2024-09-13T07:12:30.148562+0000","last_became_peered":"2024-09-13T07:12:30.148562+0000","last_unstale":"2024-09-13T07:14:59.746078+0000","last_undegraded":"2024-09-13T07:14:59.746078+0000","last_fullsized":"2024-09-13T07:14:59.746078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'677","last_scrub_stamp":"2024-09-13T07:14:59.746054+0000","last_deep_scrub":"127'677","last_deep_scrub_stamp":"2024-09-13T07:14:59.746054+0000","last_clean_scrub_stamp":"2024-09-13T07:14:59.746054+0000","objects_scrubbed":18,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:18.405899+0000","scrub_duration":0.014004912,"objects_trimmed":0,"snaptrim_duration":0.00087011300000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4449,"num_read_kb":15106,"num_write":889,"num_write_kb":8730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"127'526","reported_seq":3693,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:46.747863+0000","last_change":"2024-09-13T07:14:46.747863+0000","last_active":"2024-09-13T07:14:46.747863+0000","last_peered":"2024-09-13T07:14:46.747863+0000","last_clean":"2024-09-13T07:14:46.747863+0000","last_became_active":"2024-09-13T07:12:30.147641+0000","last_became_peered":"2024-09-13T07:12:30.147641+0000","last_unstale":"2024-09-13T07:14:46.747863+0000","last_undegraded":"2024-09-13T07:14:46.747863+0000","last_fullsized":"2024-09-13T07:14:46.747863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'526","last_scrub_stamp":"2024-09-13T07:14:46.747706+0000","last_deep_scrub":"127'526","last_deep_scrub_stamp":"2024-09-13T07:14:46.747706+0000","last_clean_scrub_stamp":"2024-09-13T07:14:46.747706+0000","objects_scrubbed":18,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:29:48.192131+0000","scrub_duration":0.01319126,"objects_trimmed":0,"snaptrim_duration":0.00085892599999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":4036,"num_read_kb":23342,"num_write":637,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"127'1136","reported_seq":4789,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:43.880407+0000","last_change":"2024-09-13T07:14:43.880407+0000","last_active":"2024-09-13T07:14:43.880407+0000","last_peered":"2024-09-13T07:14:43.880407+0000","last_clean":"2024-09-13T07:14:43.880407+0000","last_became_active":"2024-09-13T07:12:30.148622+0000","last_became_peered":"2024-09-13T07:12:30.148622+0000","last_unstale":"2024-09-13T07:14:43.880407+0000","last_undegraded":"2024-09-13T07:14:43.880407+0000","last_fullsized":"2024-09-13T07:14:43.880407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'1136","last_scrub_stamp":"2024-09-13T07:14:43.880219+0000","last_deep_scrub":"127'1136","last_deep_scrub_stamp":"2024-09-13T07:14:43.880219+0000","last_clean_scrub_stamp":"2024-09-13T07:14:43.880219+0000","objects_scrubbed":20,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:40.036943+0000","scrub_duration":0.017069734999999999,"objects_trimmed":0,"snaptrim_duration":0.00085437600000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8188,"num_read_kb":35727,"num_write":1536,"num_write_kb":7835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"127'372","reported_seq":4452,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:45.514893+0000","last_change":"2024-09-13T07:14:45.514893+0000","last_active":"2024-09-13T07:14:45.514893+0000","last_peered":"2024-09-13T07:14:45.514893+0000","last_clean":"2024-09-13T07:14:45.514893+0000","last_became_active":"2024-09-13T07:12:30.150627+0000","last_became_peered":"2024-09-13T07:12:30.150627+0000","last_unstale":"2024-09-13T07:14:45.514893+0000","last_undegraded":"2024-09-13T07:14:45.514893+0000","last_fullsized":"2024-09-13T07:14:45.514893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'372","last_scrub_stamp":"2024-09-13T07:14:45.514673+0000","last_deep_scrub":"127'372","last_deep_scrub_stamp":"2024-09-13T07:14:45.514673+0000","last_clean_scrub_stamp":"2024-09-13T07:14:45.514673+0000","objects_scrubbed":15,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:48.312032+0000","scrub_duration":0.013833412999999999,"objects_trimmed":0,"snaptrim_duration":0.00057085600000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":4499,"num_read_kb":23044,"num_write":315,"num_write_kb":7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"127'368","reported_seq":5343,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:41.673532+0000","last_change":"2024-09-13T07:14:41.673532+0000","last_active":"2024-09-13T07:14:41.673532+0000","last_peered":"2024-09-13T07:14:41.673532+0000","last_clean":"2024-09-13T07:14:41.673532+0000","last_became_active":"2024-09-13T07:12:30.147561+0000","last_became_peered":"2024-09-13T07:12:30.147561+0000","last_unstale":"2024-09-13T07:14:41.673532+0000","last_undegraded":"2024-09-13T07:14:41.673532+0000","last_fullsized":"2024-09-13T07:14:41.673532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'368","last_scrub_stamp":"2024-09-13T07:14:41.673414+0000","last_deep_scrub":"127'368","last_deep_scrub_stamp":"2024-09-13T07:14:41.673414+0000","last_clean_scrub_stamp":"2024-09-13T07:14:41.673414+0000","objects_scrubbed":14,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:11:22.900037+0000","scrub_duration":0.0088143490000000008,"objects_trimmed":0,"snaptrim_duration":0.0014230880000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5043,"num_read_kb":22928,"num_write":372,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"127'595","reported_seq":5628,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:48.515146+0000","last_change":"2024-09-13T07:14:48.515146+0000","last_active":"2024-09-13T07:14:48.515146+0000","last_peered":"2024-09-13T07:14:48.515146+0000","last_clean":"2024-09-13T07:14:48.515146+0000","last_became_active":"2024-09-13T07:12:30.143686+0000","last_became_peered":"2024-09-13T07:12:30.143686+0000","last_unstale":"2024-09-13T07:14:48.515146+0000","last_undegraded":"2024-09-13T07:14:48.515146+0000","last_fullsized":"2024-09-13T07:14:48.515146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'595","last_scrub_stamp":"2024-09-13T07:14:48.514960+0000","last_deep_scrub":"127'595","last_deep_scrub_stamp":"2024-09-13T07:14:48.514960+0000","last_clean_scrub_stamp":"2024-09-13T07:14:48.514960+0000","objects_scrubbed":23,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:33:47.773675+0000","scrub_duration":0.017592561999999999,"objects_trimmed":0,"snaptrim_duration":0.0010660229999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7688,"num_read_kb":29062,"num_write":588,"num_write_kb":8210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"127'510","reported_seq":3154,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:40.890351+0000","last_change":"2024-09-13T07:14:40.890351+0000","last_active":"2024-09-13T07:14:40.890351+0000","last_peered":"2024-09-13T07:14:40.890351+0000","last_clean":"2024-09-13T07:14:40.890351+0000","last_became_active":"2024-09-13T07:12:30.145067+0000","last_became_peered":"2024-09-13T07:12:30.145067+0000","last_unstale":"2024-09-13T07:14:40.890351+0000","last_undegraded":"2024-09-13T07:14:40.890351+0000","last_fullsized":"2024-09-13T07:14:40.890351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'510","last_scrub_stamp":"2024-09-13T07:14:40.890238+0000","last_deep_scrub":"127'510","last_deep_scrub_stamp":"2024-09-13T07:14:40.890238+0000","last_clean_scrub_stamp":"2024-09-13T07:14:40.890238+0000","objects_scrubbed":11,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:48.017258+0000","scrub_duration":0.012146716,"objects_trimmed":0,"snaptrim_duration":0.00088762699999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4290,"num_read_kb":16557,"num_write":472,"num_write_kb":7161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"127'269","reported_seq":1407,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:52.747971+0000","last_change":"2024-09-13T07:14:52.747971+0000","last_active":"2024-09-13T07:14:52.747971+0000","last_peered":"2024-09-13T07:14:52.747971+0000","last_clean":"2024-09-13T07:14:52.747971+0000","last_became_active":"2024-09-13T07:13:14.360594+0000","last_became_peered":"2024-09-13T07:13:14.360594+0000","last_unstale":"2024-09-13T07:14:52.747971+0000","last_undegraded":"2024-09-13T07:14:52.747971+0000","last_fullsized":"2024-09-13T07:14:52.747971+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'269","last_scrub_stamp":"2024-09-13T07:14:52.747805+0000","last_deep_scrub":"127'269","last_deep_scrub_stamp":"2024-09-13T07:14:52.747805+0000","last_clean_scrub_stamp":"2024-09-13T07:14:52.747805+0000","objects_scrubbed":21,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:21:59.165280+0000","scrub_duration":0.015694957999999998,"objects_trimmed":0,"snaptrim_duration":0.00073831400000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":843,"num_read_kb":7353,"num_write":220,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2494472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"127'352","reported_seq":4700,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:15:11.653666+0000","last_change":"2024-09-13T07:15:11.653797+0000","last_active":"2024-09-13T07:15:11.653666+0000","last_peered":"2024-09-13T07:15:11.653666+0000","last_clean":"2024-09-13T07:15:11.653666+0000","last_became_active":"2024-09-13T07:13:17.372133+0000","last_became_peered":"2024-09-13T07:13:17.372133+0000","last_unstale":"2024-09-13T07:15:11.653666+0000","last_undegraded":"2024-09-13T07:15:11.653666+0000","last_fullsized":"2024-09-13T07:15:11.653666+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'352","last_scrub_stamp":"2024-09-13T07:15:11.653642+0000","last_deep_scrub":"127'352","last_deep_scrub_stamp":"2024-09-13T07:15:11.653642+0000","last_clean_scrub_stamp":"2024-09-13T07:15:11.653642+0000","objects_scrubbed":15,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:18.847615+0000","scrub_duration":0.011058439,"objects_trimmed":0,"snaptrim_duration":0.001742785,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4251,"num_read_kb":14694,"num_write":400,"num_write_kb":7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1604368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"127'408","reported_seq":2138,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:49.578230+0000","last_change":"2024-09-13T07:14:49.578387+0000","last_active":"2024-09-13T07:14:49.578230+0000","last_peered":"2024-09-13T07:14:49.578230+0000","last_clean":"2024-09-13T07:14:49.578230+0000","last_became_active":"2024-09-13T07:13:17.373508+0000","last_became_peered":"2024-09-13T07:13:17.373508+0000","last_unstale":"2024-09-13T07:14:49.578230+0000","last_undegraded":"2024-09-13T07:14:49.578230+0000","last_fullsized":"2024-09-13T07:14:49.578230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'408","last_scrub_stamp":"2024-09-13T07:14:49.578192+0000","last_deep_scrub":"127'408","last_deep_scrub_stamp":"2024-09-13T07:14:49.578192+0000","last_clean_scrub_stamp":"2024-09-13T07:14:49.578192+0000","objects_scrubbed":5,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:11.168960+0000","scrub_duration":0.0096498039999999997,"objects_trimmed":0,"snaptrim_duration":0.00084238300000000004,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2560,"num_read_kb":13818,"num_write":434,"num_write_kb":4133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":796072,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":136,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.275696+0000","last_change":"2024-09-13T07:14:42.275862+0000","last_active":"2024-09-13T07:14:42.275696+0000","last_peered":"2024-09-13T07:14:42.275696+0000","last_clean":"2024-09-13T07:14:42.275696+0000","last_became_active":"2024-09-13T07:12:20.592352+0000","last_became_peered":"2024-09-13T07:12:20.592352+0000","last_unstale":"2024-09-13T07:14:42.275696+0000","last_undegraded":"2024-09-13T07:14:42.275696+0000","last_fullsized":"2024-09-13T07:14:42.275696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:42.275675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:42.275675+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.275675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:33:04.922579+0000","scrub_duration":0.0018022450000000001,"objects_trimmed":0,"snaptrim_duration":0.00039294999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"127'450","reported_seq":3865,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:50.731334+0000","last_change":"2024-09-13T07:14:50.731453+0000","last_active":"2024-09-13T07:14:50.731334+0000","last_peered":"2024-09-13T07:14:50.731334+0000","last_clean":"2024-09-13T07:14:50.731334+0000","last_became_active":"2024-09-13T07:12:30.152318+0000","last_became_peered":"2024-09-13T07:12:30.152318+0000","last_unstale":"2024-09-13T07:14:50.731334+0000","last_undegraded":"2024-09-13T07:14:50.731334+0000","last_fullsized":"2024-09-13T07:14:50.731334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'450","last_scrub_stamp":"2024-09-13T07:14:50.731308+0000","last_deep_scrub":"127'450","last_deep_scrub_stamp":"2024-09-13T07:14:50.731308+0000","last_clean_scrub_stamp":"2024-09-13T07:14:50.731308+0000","objects_scrubbed":16,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:26.717623+0000","scrub_duration":0.010798248,"objects_trimmed":0,"snaptrim_duration":0.00080608199999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4959,"num_read_kb":16241,"num_write":478,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":136,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:41.301543+0000","last_change":"2024-09-13T07:14:41.301645+0000","last_active":"2024-09-13T07:14:41.301543+0000","last_peered":"2024-09-13T07:14:41.301543+0000","last_clean":"2024-09-13T07:14:41.301543+0000","last_became_active":"2024-09-13T07:12:20.594872+0000","last_became_peered":"2024-09-13T07:12:20.594872+0000","last_unstale":"2024-09-13T07:14:41.301543+0000","last_undegraded":"2024-09-13T07:14:41.301543+0000","last_fullsized":"2024-09-13T07:14:41.301543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:41.301521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:41.301521+0000","last_clean_scrub_stamp":"2024-09-13T07:14:41.301521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:54.722336+0000","scrub_duration":0.001478416,"objects_trimmed":0,"snaptrim_duration":0.000401493,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"127'556","reported_seq":4396,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:44.649207+0000","last_change":"2024-09-13T07:14:44.649318+0000","last_active":"2024-09-13T07:14:44.649207+0000","last_peered":"2024-09-13T07:14:44.649207+0000","last_clean":"2024-09-13T07:14:44.649207+0000","last_became_active":"2024-09-13T07:12:30.464341+0000","last_became_peered":"2024-09-13T07:12:30.464341+0000","last_unstale":"2024-09-13T07:14:44.649207+0000","last_undegraded":"2024-09-13T07:14:44.649207+0000","last_fullsized":"2024-09-13T07:14:44.649207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'556","last_scrub_stamp":"2024-09-13T07:14:44.649184+0000","last_deep_scrub":"127'556","last_deep_scrub_stamp":"2024-09-13T07:14:44.649184+0000","last_clean_scrub_stamp":"2024-09-13T07:14:44.649184+0000","objects_scrubbed":12,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:38.078340+0000","scrub_duration":0.010280350000000001,"objects_trimmed":0,"snaptrim_duration":0.00075343399999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5292,"num_read_kb":14151,"num_write":656,"num_write_kb":8127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":135,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:43.514132+0000","last_change":"2024-09-13T07:14:43.514273+0000","last_active":"2024-09-13T07:14:43.514132+0000","last_peered":"2024-09-13T07:14:43.514132+0000","last_clean":"2024-09-13T07:14:43.514132+0000","last_became_active":"2024-09-13T07:12:20.753698+0000","last_became_peered":"2024-09-13T07:12:20.753698+0000","last_unstale":"2024-09-13T07:14:43.514132+0000","last_undegraded":"2024-09-13T07:14:43.514132+0000","last_fullsized":"2024-09-13T07:14:43.514132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:43.514110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:43.514110+0000","last_clean_scrub_stamp":"2024-09-13T07:14:43.514110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:17:56.064096+0000","scrub_duration":0.0016024590000000001,"objects_trimmed":0,"snaptrim_duration":0.00032896999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"127'393","reported_seq":1017,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:45.519244+0000","last_change":"2024-09-13T07:14:45.519359+0000","last_active":"2024-09-13T07:14:45.519244+0000","last_peered":"2024-09-13T07:14:45.519244+0000","last_clean":"2024-09-13T07:14:45.519244+0000","last_became_active":"2024-09-13T07:13:17.371500+0000","last_became_peered":"2024-09-13T07:13:17.371500+0000","last_unstale":"2024-09-13T07:14:45.519244+0000","last_undegraded":"2024-09-13T07:14:45.519244+0000","last_fullsized":"2024-09-13T07:14:45.519244+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'393","last_scrub_stamp":"2024-09-13T07:14:45.519221+0000","last_deep_scrub":"127'393","last_deep_scrub_stamp":"2024-09-13T07:14:45.519221+0000","last_clean_scrub_stamp":"2024-09-13T07:14:45.519221+0000","objects_scrubbed":18,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:33.482832+0000","scrub_duration":0.0124332,"objects_trimmed":0,"snaptrim_duration":0.001169427,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":1167,"num_read_kb":5375,"num_write":465,"num_write_kb":2919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":783508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":138,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:40.446065+0000","last_change":"2024-09-13T07:14:40.446156+0000","last_active":"2024-09-13T07:14:40.446065+0000","last_peered":"2024-09-13T07:14:40.446065+0000","last_clean":"2024-09-13T07:14:40.446065+0000","last_became_active":"2024-09-13T07:12:20.588850+0000","last_became_peered":"2024-09-13T07:12:20.588850+0000","last_unstale":"2024-09-13T07:14:40.446065+0000","last_undegraded":"2024-09-13T07:14:40.446065+0000","last_fullsized":"2024-09-13T07:14:40.446065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:40.446047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:40.446047+0000","last_clean_scrub_stamp":"2024-09-13T07:14:40.446047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:54.143634+0000","scrub_duration":0.0018531439999999999,"objects_trimmed":0,"snaptrim_duration":0.000292245,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"127'398","reported_seq":7875,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.885662+0000","last_change":"2024-09-13T07:14:42.885836+0000","last_active":"2024-09-13T07:14:42.885662+0000","last_peered":"2024-09-13T07:14:42.885662+0000","last_clean":"2024-09-13T07:14:42.885662+0000","last_became_active":"2024-09-13T07:12:30.141785+0000","last_became_peered":"2024-09-13T07:12:30.141785+0000","last_unstale":"2024-09-13T07:14:42.885662+0000","last_undegraded":"2024-09-13T07:14:42.885662+0000","last_fullsized":"2024-09-13T07:14:42.885662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'398","last_scrub_stamp":"2024-09-13T07:14:42.885638+0000","last_deep_scrub":"127'398","last_deep_scrub_stamp":"2024-09-13T07:14:42.885638+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.885638+0000","objects_scrubbed":12,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:33.170148+0000","scrub_duration":0.012168109999999999,"objects_trimmed":0,"snaptrim_duration":0.00065204499999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7494,"num_read_kb":20728,"num_write":374,"num_write_kb":18766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"127'608","reported_seq":4805,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:39.872464+0000","last_change":"2024-09-13T07:14:39.872567+0000","last_active":"2024-09-13T07:14:39.872464+0000","last_peered":"2024-09-13T07:14:39.872464+0000","last_clean":"2024-09-13T07:14:39.872464+0000","last_became_active":"2024-09-13T07:12:30.149020+0000","last_became_peered":"2024-09-13T07:12:30.149020+0000","last_unstale":"2024-09-13T07:14:39.872464+0000","last_undegraded":"2024-09-13T07:14:39.872464+0000","last_fullsized":"2024-09-13T07:14:39.872464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'608","last_scrub_stamp":"2024-09-13T07:14:39.872444+0000","last_deep_scrub":"127'608","last_deep_scrub_stamp":"2024-09-13T07:14:39.872444+0000","last_clean_scrub_stamp":"2024-09-13T07:14:39.872444+0000","objects_scrubbed":13,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:41.830143+0000","scrub_duration":0.0088363620000000004,"objects_trimmed":0,"snaptrim_duration":0.00098842999999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5602,"num_read_kb":29743,"num_write":825,"num_write_kb":11473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"127'119","reported_seq":258,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:57.644941+0000","last_change":"2024-09-13T07:14:57.644941+0000","last_active":"2024-09-13T07:14:57.644941+0000","last_peered":"2024-09-13T07:14:57.644941+0000","last_clean":"2024-09-13T07:14:57.644941+0000","last_became_active":"2024-09-13T07:13:14.355096+0000","last_became_peered":"2024-09-13T07:13:14.355096+0000","last_unstale":"2024-09-13T07:14:57.644941+0000","last_undegraded":"2024-09-13T07:14:57.644941+0000","last_fullsized":"2024-09-13T07:14:57.644941+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'119","last_scrub_stamp":"2024-09-13T07:14:57.644795+0000","last_deep_scrub":"127'119","last_deep_scrub_stamp":"2024-09-13T07:14:57.644795+0000","last_clean_scrub_stamp":"2024-09-13T07:14:57.644795+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:37.028404+0000","scrub_duration":0.0046721560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"127'2","reported_seq":141,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:39.700458+0000","last_change":"2024-09-13T07:14:39.700576+0000","last_active":"2024-09-13T07:14:39.700458+0000","last_peered":"2024-09-13T07:14:39.700458+0000","last_clean":"2024-09-13T07:14:39.700458+0000","last_became_active":"2024-09-13T07:12:20.593608+0000","last_became_peered":"2024-09-13T07:12:20.593608+0000","last_unstale":"2024-09-13T07:14:39.700458+0000","last_undegraded":"2024-09-13T07:14:39.700458+0000","last_fullsized":"2024-09-13T07:14:39.700458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'2","last_scrub_stamp":"2024-09-13T07:14:39.700438+0000","last_deep_scrub":"127'2","last_deep_scrub_stamp":"2024-09-13T07:14:39.700438+0000","last_clean_scrub_stamp":"2024-09-13T07:14:39.700438+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:46.103029+0000","scrub_duration":0.003116237,"objects_trimmed":0,"snaptrim_duration":0.00036623599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"127'478","reported_seq":2732,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:53.585170+0000","last_change":"2024-09-13T07:14:53.585302+0000","last_active":"2024-09-13T07:14:53.585170+0000","last_peered":"2024-09-13T07:14:53.585170+0000","last_clean":"2024-09-13T07:14:53.585170+0000","last_became_active":"2024-09-13T07:12:30.464155+0000","last_became_peered":"2024-09-13T07:12:30.464155+0000","last_unstale":"2024-09-13T07:14:53.585170+0000","last_undegraded":"2024-09-13T07:14:53.585170+0000","last_fullsized":"2024-09-13T07:14:53.585170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'478","last_scrub_stamp":"2024-09-13T07:14:53.585144+0000","last_deep_scrub":"127'478","last_deep_scrub_stamp":"2024-09-13T07:14:53.585144+0000","last_clean_scrub_stamp":"2024-09-13T07:14:53.585144+0000","objects_scrubbed":11,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:35:38.289471+0000","scrub_duration":0.0086389510000000006,"objects_trimmed":0,"snaptrim_duration":0.001761669,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3124,"num_read_kb":26626,"num_write":636,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"127'3","reported_seq":118,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:15:01.700370+0000","last_change":"2024-09-13T07:15:01.700370+0000","last_active":"2024-09-13T07:15:01.700370+0000","last_peered":"2024-09-13T07:15:01.700370+0000","last_clean":"2024-09-13T07:15:01.700370+0000","last_became_active":"2024-09-13T07:13:14.354051+0000","last_became_peered":"2024-09-13T07:13:14.354051+0000","last_unstale":"2024-09-13T07:15:01.700370+0000","last_undegraded":"2024-09-13T07:15:01.700370+0000","last_fullsized":"2024-09-13T07:15:01.700370+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'3","last_scrub_stamp":"2024-09-13T07:15:01.700243+0000","last_deep_scrub":"127'3","last_deep_scrub_stamp":"2024-09-13T07:15:01.700243+0000","last_clean_scrub_stamp":"2024-09-13T07:15:01.700243+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:43:41.718821+0000","scrub_duration":0.003892885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"127'629","reported_seq":4451,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:48.605163+0000","last_change":"2024-09-13T07:14:48.605282+0000","last_active":"2024-09-13T07:14:48.605163+0000","last_peered":"2024-09-13T07:14:48.605163+0000","last_clean":"2024-09-13T07:14:48.605163+0000","last_became_active":"2024-09-13T07:12:30.150082+0000","last_became_peered":"2024-09-13T07:12:30.150082+0000","last_unstale":"2024-09-13T07:14:48.605163+0000","last_undegraded":"2024-09-13T07:14:48.605163+0000","last_fullsized":"2024-09-13T07:14:48.605163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'629","last_scrub_stamp":"2024-09-13T07:14:48.605137+0000","last_deep_scrub":"127'629","last_deep_scrub_stamp":"2024-09-13T07:14:48.605137+0000","last_clean_scrub_stamp":"2024-09-13T07:14:48.605137+0000","objects_scrubbed":23,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:13.839766+0000","scrub_duration":0.017911283,"objects_trimmed":0,"snaptrim_duration":0.00091451499999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":4821,"num_read_kb":23588,"num_write":687,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":137,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.777247+0000","last_change":"2024-09-13T07:14:42.777404+0000","last_active":"2024-09-13T07:14:42.777247+0000","last_peered":"2024-09-13T07:14:42.777247+0000","last_clean":"2024-09-13T07:14:42.777247+0000","last_became_active":"2024-09-13T07:12:20.589291+0000","last_became_peered":"2024-09-13T07:12:20.589291+0000","last_unstale":"2024-09-13T07:14:42.777247+0000","last_undegraded":"2024-09-13T07:14:42.777247+0000","last_fullsized":"2024-09-13T07:14:42.777247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:42.777224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:42.777224+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.777224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:19:03.168524+0000","scrub_duration":0.00202528,"objects_trimmed":0,"snaptrim_duration":0.00028604900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"127'507","reported_seq":5308,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:47.542067+0000","last_change":"2024-09-13T07:14:47.542189+0000","last_active":"2024-09-13T07:14:47.542067+0000","last_peered":"2024-09-13T07:14:47.542067+0000","last_clean":"2024-09-13T07:14:47.542067+0000","last_became_active":"2024-09-13T07:12:30.146272+0000","last_became_peered":"2024-09-13T07:12:30.146272+0000","last_unstale":"2024-09-13T07:14:47.542067+0000","last_undegraded":"2024-09-13T07:14:47.542067+0000","last_fullsized":"2024-09-13T07:14:47.542067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'507","last_scrub_stamp":"2024-09-13T07:14:47.542044+0000","last_deep_scrub":"127'507","last_deep_scrub_stamp":"2024-09-13T07:14:47.542044+0000","last_clean_scrub_stamp":"2024-09-13T07:14:47.542044+0000","objects_scrubbed":13,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:32.829209+0000","scrub_duration":0.010914961000000001,"objects_trimmed":0,"snaptrim_duration":0.00066887700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5921,"num_read_kb":26865,"num_write":662,"num_write_kb":28894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:51.673305+0000","last_change":"2024-09-13T07:14:51.673305+0000","last_active":"2024-09-13T07:14:51.673305+0000","last_peered":"2024-09-13T07:14:51.673305+0000","last_clean":"2024-09-13T07:14:51.673305+0000","last_became_active":"2024-09-13T07:13:14.353467+0000","last_became_peered":"2024-09-13T07:13:14.353467+0000","last_unstale":"2024-09-13T07:14:51.673305+0000","last_undegraded":"2024-09-13T07:14:51.673305+0000","last_fullsized":"2024-09-13T07:14:51.673305+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:14:51.673180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:14:51.673180+0000","last_clean_scrub_stamp":"2024-09-13T07:14:51.673180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:12:10.822873+0000","scrub_duration":0.0022204659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"127'489","reported_seq":2399,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:51.450943+0000","last_change":"2024-09-13T07:14:51.451077+0000","last_active":"2024-09-13T07:14:51.450943+0000","last_peered":"2024-09-13T07:14:51.450943+0000","last_clean":"2024-09-13T07:14:51.450943+0000","last_became_active":"2024-09-13T07:14:13.313677+0000","last_became_peered":"2024-09-13T07:14:13.313677+0000","last_unstale":"2024-09-13T07:14:51.450943+0000","last_undegraded":"2024-09-13T07:14:51.450943+0000","last_fullsized":"2024-09-13T07:14:51.450943+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'489","last_scrub_stamp":"2024-09-13T07:14:51.450918+0000","last_deep_scrub":"127'489","last_deep_scrub_stamp":"2024-09-13T07:14:51.450918+0000","last_clean_scrub_stamp":"2024-09-13T07:14:51.450918+0000","objects_scrubbed":13,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:36.411832+0000","scrub_duration":0.010254340000000001,"objects_trimmed":0,"snaptrim_duration":0.001118324,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2849,"num_read_kb":13316,"num_write":534,"num_write_kb":9275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":3089567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"127'532","reported_seq":1550,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:54.609682+0000","last_change":"2024-09-13T07:14:54.609798+0000","last_active":"2024-09-13T07:14:54.609682+0000","last_peered":"2024-09-13T07:14:54.609682+0000","last_clean":"2024-09-13T07:14:54.609682+0000","last_became_active":"2024-09-13T07:12:30.464352+0000","last_became_peered":"2024-09-13T07:12:30.464352+0000","last_unstale":"2024-09-13T07:14:54.609682+0000","last_undegraded":"2024-09-13T07:14:54.609682+0000","last_fullsized":"2024-09-13T07:14:54.609682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'532","last_scrub_stamp":"2024-09-13T07:14:54.609659+0000","last_deep_scrub":"127'532","last_deep_scrub_stamp":"2024-09-13T07:14:54.609659+0000","last_clean_scrub_stamp":"2024-09-13T07:14:54.609659+0000","objects_scrubbed":12,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:09:42.461520+0000","scrub_duration":0.0088661380000000008,"objects_trimmed":0,"snaptrim_duration":0.00068194200000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2295,"num_read_kb":8211,"num_write":766,"num_write_kb":9441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"127'841","reported_seq":3713,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:44.246190+0000","last_change":"2024-09-13T07:14:44.246315+0000","last_active":"2024-09-13T07:14:44.246190+0000","last_peered":"2024-09-13T07:14:44.246190+0000","last_clean":"2024-09-13T07:14:44.246190+0000","last_became_active":"2024-09-13T07:12:30.148476+0000","last_became_peered":"2024-09-13T07:12:30.148476+0000","last_unstale":"2024-09-13T07:14:44.246190+0000","last_undegraded":"2024-09-13T07:14:44.246190+0000","last_fullsized":"2024-09-13T07:14:44.246190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'841","last_scrub_stamp":"2024-09-13T07:14:44.246165+0000","last_deep_scrub":"127'841","last_deep_scrub_stamp":"2024-09-13T07:14:44.246165+0000","last_clean_scrub_stamp":"2024-09-13T07:14:44.246165+0000","objects_scrubbed":17,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:13.210054+0000","scrub_duration":0.013817772000000001,"objects_trimmed":0,"snaptrim_duration":0.000444733,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":6699,"num_read_kb":19701,"num_write":999,"num_write_kb":9390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"127'870","reported_seq":3947,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:42.418304+0000","last_change":"2024-09-13T07:14:42.418304+0000","last_active":"2024-09-13T07:14:42.418304+0000","last_peered":"2024-09-13T07:14:42.418304+0000","last_clean":"2024-09-13T07:14:42.418304+0000","last_became_active":"2024-09-13T07:12:30.146876+0000","last_became_peered":"2024-09-13T07:12:30.146876+0000","last_unstale":"2024-09-13T07:14:42.418304+0000","last_undegraded":"2024-09-13T07:14:42.418304+0000","last_fullsized":"2024-09-13T07:14:42.418304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'870","last_scrub_stamp":"2024-09-13T07:14:42.418117+0000","last_deep_scrub":"127'870","last_deep_scrub_stamp":"2024-09-13T07:14:42.418117+0000","last_clean_scrub_stamp":"2024-09-13T07:14:42.418117+0000","objects_scrubbed":9,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:15.770471+0000","scrub_duration":0.0088902559999999992,"objects_trimmed":0,"snaptrim_duration":0.00085542799999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7625,"num_read_kb":20260,"num_write":988,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"127'331","reported_seq":2429,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:54.750111+0000","last_change":"2024-09-13T07:14:54.750234+0000","last_active":"2024-09-13T07:14:54.750111+0000","last_peered":"2024-09-13T07:14:54.750111+0000","last_clean":"2024-09-13T07:14:54.750111+0000","last_became_active":"2024-09-13T07:12:30.152397+0000","last_became_peered":"2024-09-13T07:12:30.152397+0000","last_unstale":"2024-09-13T07:14:54.750111+0000","last_undegraded":"2024-09-13T07:14:54.750111+0000","last_fullsized":"2024-09-13T07:14:54.750111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'331","last_scrub_stamp":"2024-09-13T07:14:54.750088+0000","last_deep_scrub":"127'331","last_deep_scrub_stamp":"2024-09-13T07:14:54.750088+0000","last_clean_scrub_stamp":"2024-09-13T07:14:54.750088+0000","objects_scrubbed":12,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:58.081201+0000","scrub_duration":0.0086890779999999994,"objects_trimmed":0,"snaptrim_duration":0.00051707100000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2384,"num_read_kb":20226,"num_write":378,"num_write_kb":12265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"127'397","reported_seq":2492,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:15:05.594809+0000","last_change":"2024-09-13T07:15:05.594973+0000","last_active":"2024-09-13T07:15:05.594809+0000","last_peered":"2024-09-13T07:15:05.594809+0000","last_clean":"2024-09-13T07:15:05.594809+0000","last_became_active":"2024-09-13T07:13:17.374207+0000","last_became_peered":"2024-09-13T07:13:17.374207+0000","last_unstale":"2024-09-13T07:15:05.594809+0000","last_undegraded":"2024-09-13T07:15:05.594809+0000","last_fullsized":"2024-09-13T07:15:05.594809+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'397","last_scrub_stamp":"2024-09-13T07:15:05.594787+0000","last_deep_scrub":"127'397","last_deep_scrub_stamp":"2024-09-13T07:15:05.594787+0000","last_clean_scrub_stamp":"2024-09-13T07:15:05.594787+0000","objects_scrubbed":18,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:33.569012+0000","scrub_duration":0.016224648000000001,"objects_trimmed":0,"snaptrim_duration":0.0014470069999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2474,"num_read_kb":8520,"num_write":349,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1654784,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"127'722","reported_seq":2143,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:15:03.661554+0000","last_change":"2024-09-13T07:15:03.661694+0000","last_active":"2024-09-13T07:15:03.661554+0000","last_peered":"2024-09-13T07:15:03.661554+0000","last_clean":"2024-09-13T07:15:03.661554+0000","last_became_active":"2024-09-13T07:13:14.355297+0000","last_became_peered":"2024-09-13T07:13:14.355297+0000","last_unstale":"2024-09-13T07:15:03.661554+0000","last_undegraded":"2024-09-13T07:15:03.661554+0000","last_fullsized":"2024-09-13T07:15:03.661554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'722","last_scrub_stamp":"2024-09-13T07:15:03.661530+0000","last_deep_scrub":"127'722","last_deep_scrub_stamp":"2024-09-13T07:15:03.661530+0000","last_clean_scrub_stamp":"2024-09-13T07:15:03.661530+0000","objects_scrubbed":14,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:26:36.377624+0000","scrub_duration":0.010893982,"objects_trimmed":0,"snaptrim_duration":0.0014177219999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6164,"num_read_kb":25583,"num_write":834,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":9822921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"127'1078","reported_seq":7561,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:15:13.724738+0000","last_change":"2024-09-13T07:15:13.724897+0000","last_active":"2024-09-13T07:15:13.724738+0000","last_peered":"2024-09-13T07:15:13.724738+0000","last_clean":"2024-09-13T07:15:13.724738+0000","last_became_active":"2024-09-13T07:13:17.373942+0000","last_became_peered":"2024-09-13T07:13:17.373942+0000","last_unstale":"2024-09-13T07:15:13.724738+0000","last_undegraded":"2024-09-13T07:15:13.724738+0000","last_fullsized":"2024-09-13T07:15:13.724738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'1078","last_scrub_stamp":"2024-09-13T07:15:13.724697+0000","last_deep_scrub":"127'1078","last_deep_scrub_stamp":"2024-09-13T07:15:13.724697+0000","last_clean_scrub_stamp":"2024-09-13T07:15:13.724697+0000","objects_scrubbed":17,"log_size":1078,"log_dups_size":0,"ondisk_log_size":1078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:39:33.100096+0000","scrub_duration":0.016211963999999999,"objects_trimmed":0,"snaptrim_duration":0.0018446199999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":10238,"num_read_kb":17178,"num_write":1683,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4488815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"127'465","reported_seq":2553,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:45.207709+0000","last_change":"2024-09-13T07:14:45.207709+0000","last_active":"2024-09-13T07:14:45.207709+0000","last_peered":"2024-09-13T07:14:45.207709+0000","last_clean":"2024-09-13T07:14:45.207709+0000","last_became_active":"2024-09-13T07:12:30.149578+0000","last_became_peered":"2024-09-13T07:12:30.149578+0000","last_unstale":"2024-09-13T07:14:45.207709+0000","last_undegraded":"2024-09-13T07:14:45.207709+0000","last_fullsized":"2024-09-13T07:14:45.207709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'465","last_scrub_stamp":"2024-09-13T07:14:45.207570+0000","last_deep_scrub":"127'465","last_deep_scrub_stamp":"2024-09-13T07:14:45.207570+0000","last_clean_scrub_stamp":"2024-09-13T07:14:45.207570+0000","objects_scrubbed":18,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:59.376286+0000","scrub_duration":0.016652568,"objects_trimmed":0,"snaptrim_duration":0.00042662800000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3233,"num_read_kb":21310,"num_write":487,"num_write_kb":27814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"127'481","reported_seq":3933,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:15:10.619256+0000","last_change":"2024-09-13T07:15:10.619402+0000","last_active":"2024-09-13T07:15:10.619256+0000","last_peered":"2024-09-13T07:15:10.619256+0000","last_clean":"2024-09-13T07:15:10.619256+0000","last_became_active":"2024-09-13T07:13:17.373653+0000","last_became_peered":"2024-09-13T07:13:17.373653+0000","last_unstale":"2024-09-13T07:15:10.619256+0000","last_undegraded":"2024-09-13T07:15:10.619256+0000","last_fullsized":"2024-09-13T07:15:10.619256+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'481","last_scrub_stamp":"2024-09-13T07:15:10.619233+0000","last_deep_scrub":"127'481","last_deep_scrub_stamp":"2024-09-13T07:15:10.619233+0000","last_clean_scrub_stamp":"2024-09-13T07:15:10.619233+0000","objects_scrubbed":17,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:43.585583+0000","scrub_duration":0.012657977000000001,"objects_trimmed":0,"snaptrim_duration":0.00056591900000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4025,"num_read_kb":31952,"num_write":558,"num_write_kb":12321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":3113520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"127'312","reported_seq":1926,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:14:44.861899+0000","last_change":"2024-09-13T07:14:44.861899+0000","last_active":"2024-09-13T07:14:44.861899+0000","last_peered":"2024-09-13T07:14:44.861899+0000","last_clean":"2024-09-13T07:14:44.861899+0000","last_became_active":"2024-09-13T07:13:17.373364+0000","last_became_peered":"2024-09-13T07:13:17.373364+0000","last_unstale":"2024-09-13T07:14:44.861899+0000","last_undegraded":"2024-09-13T07:14:44.861899+0000","last_fullsized":"2024-09-13T07:14:44.861899+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'312","last_scrub_stamp":"2024-09-13T07:14:44.861723+0000","last_deep_scrub":"127'312","last_deep_scrub_stamp":"2024-09-13T07:14:44.861723+0000","last_clean_scrub_stamp":"2024-09-13T07:14:44.861723+0000","objects_scrubbed":21,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:23.458200+0000","scrub_duration":0.013567737,"objects_trimmed":0,"snaptrim_duration":0.00083458600000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1303,"num_read_kb":13724,"num_write":239,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":3518141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"127'702","reported_seq":4195,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T07:15:06.576617+0000","last_change":"2024-09-13T07:15:06.576741+0000","last_active":"2024-09-13T07:15:06.576617+0000","last_peered":"2024-09-13T07:15:06.576617+0000","last_clean":"2024-09-13T07:15:06.576617+0000","last_became_active":"2024-09-13T07:13:17.373826+0000","last_became_peered":"2024-09-13T07:13:17.373826+0000","last_unstale":"2024-09-13T07:15:06.576617+0000","last_undegraded":"2024-09-13T07:15:06.576617+0000","last_fullsized":"2024-09-13T07:15:06.576617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'702","last_scrub_stamp":"2024-09-13T07:15:06.576594+0000","last_deep_scrub":"127'702","last_deep_scrub_stamp":"2024-09-13T07:15:06.576594+0000","last_clean_scrub_stamp":"2024-09-13T07:15:06.576594+0000","objects_scrubbed":12,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:19:50.669310+0000","scrub_duration":0.010149431,"objects_trimmed":0,"snaptrim_duration":0.000482919,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5731,"num_read_kb":24818,"num_write":791,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2045382,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":501,"num_read":148029,"num_read_kb":627646,"num_write":19914,"num_write_kb":356576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":33411550,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0},"log_size":17227,"ondisk_log_size":17227,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17040,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266715,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42512,"kb_used_data":1544,"kb_used_omap":9,"kb_used_meta":40950,"kb_avail":94329328,"statfs":{"total":96636764160,"available":96593231872,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":10093,"internal_metadata":41932947},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.441},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.501}]},{"osd":4,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40000000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.46000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.21299999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42199999999999999},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.59099999999999997}]},{"osd":1,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.35199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55500000000000005}]},{"osd":3,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.748,"15min":0.748},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.313},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.33700000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:14:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.35799999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.48499999999999999},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71399999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.40200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.248,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.37,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23799999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.307}]},{"osd":6,"last update":"Fri Sep 13 07:14:18 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.374}]},{"osd":7,"last update":"Fri Sep 13 07:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.39700000000000002},"min":{"1min":0.36799999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.42799999999999999,"15min":0.40999999999999998},"min":{"1min":0.34999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.34999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64600,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307240,"statfs":{"total":96636764160,"available":96570613760,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65900000000000003,"5min":0.878,"15min":0.878},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.27400000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.378}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"last":0.33800000000000002}]},{"osd":7,"last update":"Fri Sep 13 07:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.47399999999999998,"15min":0.46100000000000002},"min":{"1min":0.375,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"min":{"1min":0.39900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.372},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.219,"15min":0.219},"max":{"1min":1.151,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.432}]},{"osd":1,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.377,"15min":0.38400000000000001},"min":{"1min":0.248,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.42199999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.498},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.40899999999999997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0109999999999999,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.624},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63600000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.505,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:14:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.107,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1519616,"data_stored":724744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.29799999999999999},"min":{"1min":0.22500000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.255,"5min":0.191,"15min":0.191},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.499}]},{"osd":1,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33300000000000002,"15min":0.32600000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33700000000000002},"min":{"1min":0.25600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.433}]},{"osd":3,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.249,"15min":0.249},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.748}]},{"osd":6,"last update":"Fri Sep 13 07:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Sep 13 07:14:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40848,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330992,"statfs":{"total":96636764160,"available":96594935808,"internally_reserved":0,"allocated":1581056,"data_stored":1315131,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.40000000000000002,"15min":0.41099999999999998},"min":{"1min":0.218,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46600000000000003,"5min":1.821,"15min":1.821},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38300000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65400000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.441},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.75800000000000001,"5min":2.234,"15min":2.234},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":1.179,"15min":1.179},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.747}]},{"osd":5,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.347,"5min":1.859,"15min":1.859},"last":0.626},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Sep 13 07:14:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":1.913,"15min":1.913},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Sep 13 07:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58299999999999996,"15min":0.56999999999999995},"min":{"1min":0.49099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.89200000000000002,"5min":1.278,"15min":1.278},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92700000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54744,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317096,"statfs":{"total":96636764160,"available":96580706304,"internally_reserved":0,"allocated":1523712,"data_stored":724763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.58499999999999996,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.55100000000000005,"15min":0.58099999999999996},"min":{"1min":0.22,"5min":0.192,"15min":0.192},"max":{"1min":0.67400000000000004,"5min":4.54,"15min":4.54},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.42899999999999999},"min":{"1min":0.245,"5min":0.214,"15min":0.214},"max":{"1min":0.628,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.378}]},{"osd":3,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.433,"15min":0.438},"min":{"1min":0.27200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.435},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.32700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.75800000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.878,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88500000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:14:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.999,"15min":0.999},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.879},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8520,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7126,"internal_metadata":0}]}} 2024-09-13T07:15:22.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T07:15:22.640 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T07:15:22.640 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T07:15:22.640 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T07:15:22.640 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:22.685 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T07:15:22.685 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T07:15:22.685 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:22.720 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T07:15:22.721 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T07:15:22.721 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:22.795 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T07:15:22.796 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T07:15:22.796 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:22.870 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T07:15:22.870 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T07:15:22.870 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:22.921 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T07:15:22.921 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T07:15:22.921 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:22.956 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T07:15:22.956 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T07:15:22.957 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:23.034 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T07:15:23.034 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T07:15:23.034 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:23.110 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T07:15:23.110 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T07:15:23.111 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T07:15:23.111 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:23.135 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T07:15:23.135 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T07:15:23.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:23.181 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T07:15:23.181 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T07:15:23.181 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T07:15:23.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:23.192 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T07:15:23.192 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T07:15:23.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:23.243 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T07:15:23.243 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T07:15:23.244 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:15:23.256 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T07:15:23.256 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T07:15:23.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T07:15:23.322 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:15:23.323 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T07:15:23.475 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:15:23.475 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T07:15:23.555 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:15:23.555 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T07:15:23.621 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:15:23.622 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T07:15:23.671 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:15:23.672 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T07:15:23.769 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:15:23.769 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T07:15:23.872 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:15:23.872 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T07:15:23.929 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:15:23.929 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T07:15:23.983 INFO:tasks.ceph:Archiving mon data... 2024-09-13T07:15:23.984 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/data/mon.a.tgz 2024-09-13T07:15:23.984 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-13T07:15:24.000 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.xYTDZ47Sfo 2024-09-13T07:15:24.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xYTDZ47Sfo 2024-09-13T07:15:24.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.xYTDZ47Sfo 2024-09-13T07:15:24.242 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.xYTDZ47Sfo is 343KB 2024-09-13T07:15:24.340 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.xYTDZ47Sfo 2024-09-13T07:15:24.356 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/data/mon.c.tgz 2024-09-13T07:15:24.356 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-13T07:15:24.410 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.33BDf7e1H8 2024-09-13T07:15:24.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.33BDf7e1H8 2024-09-13T07:15:24.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.33BDf7e1H8 2024-09-13T07:15:24.656 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.33BDf7e1H8 is 466KB 2024-09-13T07:15:24.754 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.33BDf7e1H8 2024-09-13T07:15:24.769 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/data/mon.b.tgz 2024-09-13T07:15:24.770 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-09-13T07:15:24.789 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.b14sxWFMJW 2024-09-13T07:15:24.789 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.b14sxWFMJW 2024-09-13T07:15:24.962 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.b14sxWFMJW 2024-09-13T07:15:25.045 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.b14sxWFMJW is 465KB 2024-09-13T07:15:25.144 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.b14sxWFMJW 2024-09-13T07:15:25.160 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T07:15:25.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T07:15:25.163 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T07:15:25.225 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T07:15:25.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T07:15:25.231 DEBUG:teuthology.orchestra.run.smithi184:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T07:15:25.258 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T07:15:25.292 INFO:teuthology.orchestra.run.smithi184.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T07:15:25.297 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T07:15:25.298 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/remote/smithi031/crash 2024-09-13T07:15:25.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T07:15:25.330 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/remote/smithi184/crash 2024-09-13T07:15:25.331 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T07:15:25.360 INFO:tasks.ceph:Compressing logs... 2024-09-13T07:15:25.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:15:25.374 DEBUG:teuthology.orchestra.run.smithi184:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:15:34.102 INFO:tasks.ceph:Archiving logs... 2024-09-13T07:15:34.102 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/remote/smithi031/log 2024-09-13T07:15:34.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T07:15:35.129 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/remote/smithi184/log 2024-09-13T07:15:35.130 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T07:15:35.998 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T07:15:36.007 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T07:15:36.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T07:15:36.010 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T07:15:36.113 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T07:15:36.123 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T07:15:36.123 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T07:15:36.125 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T07:15:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-13T07:15:36.141 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-09-13T07:15:36.145 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T07:15:36.145 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-13T07:15:36.145 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +1151us[+1155us] +/- 46ms 2024-09-13T07:15:36.145 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -2311us[-2311us] +/- 66ms 2024-09-13T07:15:36.145 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +829us[ +832us] +/- 90ms 2024-09-13T07:15:36.145 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:15:36.147 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T07:15:36.147 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-09-13T07:15:36.147 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +1156us[+1158us] +/- 46ms 2024-09-13T07:15:36.147 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -2311us[-2309us] +/- 66ms 2024-09-13T07:15:36.147 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +840us[ +843us] +/- 90ms 2024-09-13T07:15:36.147 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T07:15:36.147 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T07:15:36.165 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T07:15:36.165 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T07:15:36.174 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T07:15:36.266 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T07:15:36.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T07:15:36.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:15:36.367 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T07:15:36.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:15:36.396 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T07:15:36.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T07:15:36.415 INFO:teuthology.task.internal:Duration was 599.696041 seconds 2024-09-13T07:15:36.416 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T07:15:36.424 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T07:15:36.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T07:15:36.427 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T07:15:36.465 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T07:15:36.476 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T07:15:36.881 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T07:15:36.881 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T07:15:36.881 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T07:15:36.942 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-09-13T07:15:36.942 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T07:15:36.966 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T07:15:36.966 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:15:36.986 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:15:37.033 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T07:15:37.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T07:15:37.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T07:15:37.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T07:15:37.196 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T07:15:37.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T07:15:37.222 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T07:15:37.247 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T07:15:37.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:15:37.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:15:37.286 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-13T07:15:37.310 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-09-13T07:15:37.338 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:15:37.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:15:37.365 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:15:37.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:15:37.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T07:15:37.402 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T07:15:37.403 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/remote/smithi031 2024-09-13T07:15:37.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T07:15:37.445 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900909/remote/smithi184 2024-09-13T07:15:37.445 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T07:15:37.484 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T07:15:37.484 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T07:15:37.488 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T07:15:37.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T07:15:37.552 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T07:15:37.552 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T07:15:37.621 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T07:15:37.621 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T07:15:37.624 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T07:15:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 07:15 /home/ubuntu/cephtest 2024-09-13T07:15:37.639 INFO:teuthology.orchestra.run.smithi184.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 07:15 /home/ubuntu/cephtest 2024-09-13T07:15:37.641 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T07:15:37.653 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T07:15:37.674 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} duration: 599.6960413455963 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T07:15:37.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T07:15:37.754 INFO:teuthology.run:pass